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IV 


ABSTRACT 


Reconnaissance  missions  are  not  only  one  of  the  vital  modes  of  intelligence-gathering 
methods;  they  are  one  of  the  most  important  contributors  of  military  intelligence  as  well. 
They  show  the  battlefield  as  it  is  to  the  commander. 

A  simplified  reconnaissance  cycle  includes  the  arrival  of  reconnaissance  requests, 
planning  of  reconnaissance  flights,  flying  the  mission  and  exploitation  of  the  films  or 
images,  and  then  dissemination  of  the  intelligence  reports.  The  reconnaissance  cycle  is 
modeled  for  four  different  scenarios  (peace  and  war  as  situations,  RF-4  and  F-16  as 
configurations).  There  are  two  points  of  view  regarding  this  cycle.  The  first  is  the 
reconnaissance  requesters’  view:  they  want  to  know  the  estimated  time  it  would  take  for  a 
request  to  be  answered,  based  on  the  resources  and  other  factors,  before  an  actual  request 
was  made.  The  second  is  the  reconnaissance  squadron  commanders’  perspective:  they 
want  to  respond  to  as  many  reconnaissance  requests  as  possible.  For  that  reason,  they 
want  to  know  and  revise  the  ideal  numbers  of  personnel  and  equipment.  For  the  purpose 
of  answering  these  questions,  satisfying  these  requests,  and  having  a  better  understanding 
about  the  reconnaissance  cycle,  Reconnaissance  Squadron  Workflow  is  modeled, 
experimented  and  analyzed  in  this  thesis. 

Analysis  includes  regression  models  and  partition  trees.  When  results  are 
considered,  we  see  that  there  is  no  common  rule  to  determine  which  factors  (either 
decision  or  noise)  are  the  key  determinants  for  each  scenario.  But  we  noticed  that  noise 
factors  have  much  more  impact  on  several  measures  of  effectiveness  than  decision  factors 
in  each  model.  Some  of  these  noise  factors  could  be  controllable,  including  aircraft, 
camera  and  pod  defect  probabilities  and  their  repair  times.  Therefore,  some  precautionary 
measures  should  be  taken  to  reduce  these  defect  probabilities  and  repair  times. 

Specifically,  in  the  RF-4  configuration  models,  pilot  filming  error  is  a  significant 
factor,  which  shows  that  training  of  the  pilots  cannot  be  ignored.  When  the  F-16  models 
are  considered,  we  see  that  data  link  defect  probability  is  a  significant  factor  too.  This 
suggests  that  special  precautions  should  be  taken  to  keep  this  capability  working. 
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DISCLAIMER 


The  reader  is  cautioned  that  the  computer  program  developed  in  this  research  may 
not  have  been  exercised  for  all  cases  of  interest.  While  every  effort  has  been  made,  within 
the  time  available,  to  ensure  that  the  programs  are  free  of  computational  errors,  they 
cannot  be  considered  validated.  Any  application  of  these  programs  without  additional 
verification  is  at  the  risk  of  the  planner. 
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I.  INTRODUCTION 

A.  BACKGROUND  AND  MOTIVATION 

Using  intelligence  gathering  capabilities,  military  intelligence  collects  information 
about  hostile,  friendly  and  neutral  forces.  Intelligence  activities  are  continuously 
processed  at  all  levels,  from  tactical  to  strategic,  in  peacetime,  the  period  of  transition  to 
war,  and  during  a  war  itself. 

Some  of  the  military  intelligence  gathering  capabilities  include  Human 
Intelligence  (HUMINT),  Signal  Intelligence  (SIGINT),  Open  Source  Intelligence 
(OSINT),  and  Imagery  Intelligence  (IMINT). 

The  IMINT  is  conducted  by  unmanned  or  manned  aerial  vehicles.  Those  vehicles’ 
missions  also  include  reconnaissance,  which  is  performed  over  specific  targets  at  specific 
times.  Reconnaissance  missions  are  not  just  one  of  the  intelligence  gethering  methods, 
but  one  of  the  most  important  contributors  to  military  intelligence.  Reconnaissance 
missions  are  highly  important  to  military  operations,  since  they  are  the  only  resources 
which  show  the  commander  the  battlefield  as  it  is.  Reconnaissance  missions  provide 
insight  for  decison-makers  and  they  are  the  key  for  victory. 

To  provide  IMINT,  a  wide  variety  of  sensors  and  platforms  are  in  operational  use 
in  the  theather.  Platforms  and  sensors  have  evolved  with  the  advent  of  aviation  and 
photography,  respectively.  In  the  past,  balloons,  rockets,  and  kites  were  used  as 
platforms.  Previously,  observers  in  the  balloons,  sketchers  in  the  back  of  the  planes,  and 
optical  devices  that  were  taken  into  the  air  and  handheld  were  early  sensors  used  for 
reconnaissance.  Now,  sensors  are  mounted  and  used  on  platforms  such  as  satellites, 
aircrafts,  and  unmanned  air  vehicles. 

Aircraft  have  an  advantage  over  other  platforms  since  they  can  be  deployed 
quicker  and  thus  gather  information  about  the  area  of  interest  much  faster  (Gething, 
Hewish,  &  Lok,  2003).  The  RF-4  and  F-16,  which  are  modeled  in  the  thesis,  are  two 
aircraft  that  are  used  for  the  purpose  of  reconnaissance  missions.  The  RF-4  has  a 
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specially  designed  nose  in  which  optical  sensors  can  be  mounted,  and  can  also  carry 
recconnaissance  pods,  while  the  F-16  can  do  reconnaissance  missions  with 
reconnaissance  pods  only. 

Sensors  used  in  a  reconnaissance  mission  can  be  film-based  or  Electro-Optic 
(E/0).  The  difference  between  a  wet-film-based  sensor  and  an  E/0  sensor  is  that  the  wet- 
based  sensors  need  to  be  processed  before  the  films  are  interpreted,  thus  requiring  extra 
time.  For  the  E/0  sensor,  target  images  can  be  downloaded  to  exploitation  workstations 
via  data  link  while  the  aircraft  is  still  in  the  air,  or  downloaded  directly  to  exploitation 
workstations  after  the  aircraft  lands.  As  Gething  (2008)  mentions,  this  reduces  the 
‘sensor-to-shooter  time.’  That  is,  reconnissance  reports  are  provided  faster  with  a  net- 
based  dissemination  possibility  of  digital  images,  which  is  a  great  advantage  to  the 
decision  makers  (Gething,  Hewish,  &  Lok,  2003).  In  this  thesis,  the  RF-4  is  modeled  as  a 
film-based  reconnaissance  aircraft,  and  the  F-16  as  an  E/0  pod-based  aircraft. 

Oxlee  (1997)  categorizes  reconnaissance  requirements  as  either  strategic  or 
tactical.  Strategic  reconnaissance  is  conducted  in  peacetime,  times  of  conflict,  or  during 
war.  Tactical  reconnaissance  is  conducted  during  times  of  conflict  and  in  war.  One  aspect 
of  tactical  reconnaissance  is  the  importance  of  training  during  peacetime.  Based  on  the 
“Train  like  you  fight”  principle,  reconnaissance  missions  are  simulated  inside  the  borders 
of  the  country  where  both  territory  and  targets  are  similar  to  the  real  targets.  The 
aforementioned  aircraft  types  can  carry  out  both  strategic  and  tactical  reconnaissance 
missions  with  the  appropriate  types  of  sensors  mounted. 

A  simplified  recconnaissance  cycle  includes  the  arrival  of  reconnaissance 
requests,  planning  of  reconnaissance  flights,  execution  of  the  mission  flights  and 
exploitation  of  the  films,  and  then  the  dissemination  of  the  intelligence  reports.  Although 
there  are  commonalities,  the  reconnaissance  cycle  is  different  in  peace  and  war  situations. 
Therefore,  strategic  and  tactical  reconnaissance  requirements  are  evaluated  using 
different  models  for  each  situation.  Any  delay  in  the  loop  of  providing  reconnaissance 
reports  due  to  the  shortage  of  aircrafts,  cameras,  pilots  or  image  analysts  can  cause  severe 
problems. 
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To  have  a  better  understanding  about  the  reconnaissance  system  the 
Reconnaissance  Squadron  Workflow  is  modeled.  Two  situations  are  modeled:  peace  and 
war;  and  for  each  situation  there  are  two  configurations,  namely  the  RF-4  and  the  F-16. 

B.  RELATED  RESEARCH 

One  of  the  related  thesis  research  studies  (computer  specialist  shortage  problem) 
was  conducted  by  another  Turkish  Air  Force  officer,  Serhat  Camur  (2009).  Camur 
modeled  computer  system  specialist  non-commisioned  officers’  jobs  on  a  Turkish  Air 
Force  Base  by  using  event  graph  and  discrete  event  simulation  techniques  to  determine 
the  average  time  for  repairing  an  entity  (mean  delay  time  in  system)  and  average  number 
of  entity  failures  (mean  number  in  the  queue)  waiting  for  repair.  Camur  identified  the 
factors  that  have  the  most  significant  effects  on  these  two  performance  measures  by 
making  a  nearly  orthogonal  Latin  hypercube  (NOTH)  experimental  design,  running 
simulation  for  this  design  with  100  replications,  and  conducting  statistical  analysis  on 
simulation  output  data.  Camur  concluded  that  “the  results  do  show  that  increasing  the 
staff  is  not  the  only  solution  for  his  particular  research.  There  are  some  other  factors  that 
can  be  played  with  to  decrease  the  time  in  the  system  and  mean  number  in  queue.” 

After  comparing  Camur’ s  results  to  our  thesis  research,  we  see  that  we  have  a 
similar  performance  measure  (mean  delay  time  in  system  vs.  average  time  elapsed 
between  getting  reconnaissance  requests  and  providing  reconnaissance  reports),  and 
nearly  the  same  methodology.  Camur’ s  model  is  based  on  actual  data  and  operations, 
though  our  model  is  based  on  notional  data  and  realistic  operations. 

C.  RESEARCH  QUESTIONS 

This  thesis  will  attempt  to  answer  the  following  research  questions: 

i.  Given  an  operation  scenario: 

i.  What  is  the  average  time  elapsed  between  getting  reconnaissance 
requests  and  providing  reconnaissance  reports? 
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ii.  What  is  the  ratio  of  not  responded  reeonnaissance  requests  over  total 
arrived  reconnaissanee  requests? 

ii.  What  is  the  ideal  number  of  aircrafts,  cameras,  pilots,  and  image  analysts 
(between  realistic  minimum  and  maximum  values)  to  support  a  given  operation 
scenario? 

D.  THE  SCOPE  OF  THE  THESIS 

The  primary  goal  of  this  thesis  is  to  develop  a  simulation  tool  that  will  enable 
planners  at  headquarters  to  estimate  the  measures  listed  below: 

1 .  Average  time  to  provide  a  reconnaissance  report. 

2.  Parameters  that  affect  the  elapsed  time  between  getting  reconnaissance 
requests  and  providing  reconnaissance  reports. 

3.  Usage  of  the  resources  such  as  aircrafts,  cameras,  pilots  and  image  analysts. 

4.  Bottleneck  areas  of  the  Reconnaissance  Squadron  Workflow. 

A  secondary  goal  of  this  thesis  is  to  find  the  best  configuration  (i.e.,  the  ideal 
number  of  aircraft,  cameras,  pilots,  and  image  analysts)  to  support  a  given  operation 
scenario. 

Both  a  thirty-day  war  scenario  and  a  peace  scenario  with  two  different  kinds  of 
structures  (organizational  or  configurational)  are  investigated. 

E.  METHODOLOGY 

The  methodology  used  in  this  thesis  is  as  follows: 

1 .  Conduct  research  on  current  and  past  similar  Decision  Support  Systems  and 
tools.  This  will  set  a  general  guideline  for  the  thesis  and  give  further  ideas. 

2.  Conduct  research  on  current  Reconnaissance  Squadron  Workflow  to  create  a 
realistic  model. 
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3.  Develop  and  draw  detailed  event  graph  components  for  the  discrete  event 
simulation.  These  will  set  the  basis  for  the  Simkit  implementation  of  the 
model. 

4.  Develop  a  simulation  tool  using  Java  programming  language.  That  tool  will  be 
a  DBS  implementation  using  the  event  graph  components  obtained  in  the 
previous  step. 

5.  Verify  (test  and  debug)  and  validate  (adequately  capturing  the  essence  of  the 
problem)  the  simulation  with  the  help  of  feedback,  ideas  and 
recommendations  coming  from  pilots,  planners  and  image  analysts  working  in 
the  Turkish  Air  Force. 

6.  Analyze  results.  Develop  a  methodology  on  how  to  obtain  and  analyze  the 
data  produced  by  the  tool.  The  purpose  is  to  give  a  general  guideline  on  how 
to  extract  the  needed  data  and  how  to  run  the  proper  analysis  methods. 

F.  BENEFITS  OF  THIS  STUDY 

A  technological  approach  is  used  to  assess  the  need  for  personnel  for  a  specific 
branch  (image  analysts);  this  may  be  enlarged  later  to  the  other  branches  in  the  other 
facilities  of  the  Turkish  Air  Force.  An  improved  assignment  can  be  achieved  by  using  the 
DBS  technique. 

Insight  can  be  obtained  from  the  effects  of  camera/aircraft  failures,  aircraft,  pilot 
or  image  analyst  shortages,  etc.,  when  it  comes  to  providing  a  reconnaissance  report. 
After  making  a  trade-off  study,  alterations  can  be  made  in  those  areas  to  optimize  the 
Reconnaissance  Squadron  Workflow. 
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II.  MODELING  AND  SIMULATION 


“Modeling  and  Simulation  is  a  discipline  for  developing  a  level  of  understanding 
of  the  interaction  of  the  parts  of  a  system,  and  of  the  system  as  a  whole”  (Bellinger, 
2004).  This  chapter  describes  some  of  the  major  terms  about  modeling  and  simulation 
such  as  system,  model,  and  simulation.  It  explains  the  modeling  process  in  detail.  Then  it 
describes  the  Discrete  Event  Simulation  (DES)  and  event  graph  methodology.  Finally,  it 
describes  the  Simkit  package  that  was  used  to  create  the  Reconnaissance  Squadron 
Workflow  model. 

A.  SYSTEM,  MODEL  AND  SIMULATION 

“A  system  is  defined  to  be  a  collection  of  entities,  e.g.,  people  or  machines,  which 
act  and  interact  together  toward  the  accomplishment  of  some  logical  end”  (Law,  2007). 
An  example  for  a  system  is  the  queuing  system,  which  is  made  up  of  customers,  a  queue, 
and  a  server.  Physical  entities  of  the  system  are  the  customers  and  server;  on  the  other 
hand,  the  queue  itself  is  a  concept.  All  of  the  system  entities  and  their  attributes  constitute 
the  state  of  the  system  (Sanchez,  2007). 

A  model  can  be  defined  as  the  representation  of  a  system  used  to  study  it  (Law, 
2007).  There  are  many  reasons  for  using  a  model  of  a  system  instead  of  using  a  system 
itself  For  example,  models  can  enable  us  to  study  how  a  prospective  system  will  work 
before  the  real  system  has  even  been  built.  Building  and  studying  a  model  is  only  a  small 
portion  of  the  cost  of  experimenting  with  the  real  system  in  many  cases.  A  model  has  the 
ability  to  scale  time  or  space  in  a  favorable  manner — for  example,  with  a  flight  simulator, 
wind  sheer  conditions  can  be  created  on  demand  (Sanchez,  2007).  In  addition  to  all  of 
these  reasons,  in  some  cases  a  model  becomes  a  necessity  because  it  is  very  dangerous 
and  often  impossible  to  conduct  experiments  using  real  systems. 

Since  all  models  are  simplifications  of  reality  there  is  always  a  trade-off  as 
to  what  level  of  detail  is  included  in  the  model.  If  too  little  detail  is 
included  in  the  model  one  runs  the  risk  of  missing  relevant  interactions 
and  the  resultant  model  does  not  promote  understanding.  If  too  much 
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detail  is  included  in  the  model  the  model  may  become  overly  complicated 
and  actually  preclude  the  development  of  understanding.  (Bellinger,  2004) 


There  are  many  variations  of  models.  One  type  is  physical  representations  (with 
or  without  scaling)  such  as  wind  tunnel  mockups.  Another  variation  consists  of 
mathematical  equations  such  as  the  equations  of  motion  found  in  a  typical  physics  book. 
Computer  simulations  (programs),  such  as  the  ones  used  in  modem  flight  simulators,  are 
also  a  variation  of  models  (Sanchez,  2007). 

Simulation  can  be  defined  as: 

a  computerized  version  of  the  model  which  is  mn  over  time  to  study  the 
implications  of  the  defined  interactions... Simulations  are  generally 
iterative  in  their  development.  One  develops  a  model,  simulates  it,  learns 
from  the  simulation,  revises  the  model,  and  continues  the  iterations  until 
an  adequate  level  of  understanding  is  developed.  (Bellinger,  2004) 

B.  MODELING  PROCESS 

Modeling  is  an  iterative  process  with  feedback.  It  can  be  divided  into  several 
basic  stages.  In  stage  1,  the  scope  of  the  model  is  decided  to  identify  what  is  meant  by  the 
system  of  interest.  A  descriptive  model  is  built  at  the  end  of  stage  1.  In  stage  2,  the 
behaviors  and  interactions  of  all  of  the  entities  that  comprise  the  system  are  described.  A 
formal  model  is  built  at  the  end  of  stage  2.  “If  the  formal  model  has  a  high  degree  of 
conformance  with  the  real  world  system  being  modeled,  analytic  models  and  their 
solutions  allow  us  to  obtain  insights  and  draw  inferences  about  the  real  system  as  seen 
from  Figure  1.  ”  (Sanchez,  2007). 


Inference 


Figure  1.  A  Model  Yields  Insights  and  Inferences  [From  (Sanchez,  2007)] 
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Adding  more  realistic  features  such  as  non-homogeneous  arrival  and  service  rates, 
machinery  breaking  down,  etc.,  leads  to  models  that  cannot  be  solved  analytically.  In 
many  cases  computer  simulation  can  be  created  that  describes  these  features 
algorithmically.  The  model  described  in  Chapter  III  is  an  example  of  such  a  simulation 
model.  The  resultant  simulation  model  often  uses  randomness  as  part  of  the  modeling 
process  so  its  output  becomes  a  random  variable.  For  this  reason,  statistics  enters  into  the 
process  and  a  statistical  model  of  the  computer  model  (built  from  the  formal  model)  is 
built.  The  whole  process  is  illustrated  in  Figure  2.  (Sanchez,  2007). 


Figure  2.  Simulation  has  a  Longer  Chain  of  Inference  [From  (Sanchez,  2007)] 

Feedback  enters  the  modeling  process  in  the  form  of  verification  and  validation 
(Sargent,  2003).  Verification  is  a  feedback  loop  between  the  computer  model  and  the 
formal  model  that  addresses  the  question  “does  my  computer  program  do  what  I  meant  it 
to  do?”  and  corresponds  to  the  debugging  of  computer  model.  Validation  is  a  feedback 
loop  between  the  computer  model  and  reality  that  addresses  the  question  “does  my 
computer  program  mimic  reality  adequately?”  (Sanchez,  2007) 

Sanchez  (2007)  gives  some  recommendations  to  model  developers:  start  small, 
improve  incrementally,  test  frequently  and  backtrack/simplify. 

C.  DISCRETE  EVENT  SIMULATION  AND  EVENT  GRAPHS 

Discrete  Event  Simulation  (DBS)  is  a  methodology  which  models  a  system  as  the 

state  change  occurs  at  a  discrete  set  of  points  along  the  time  axis,  rather  than  continuously 

(Sanchez,  2007).  There  are  four  basic  elements  of  a  DES  model:  states,  events, 

scheduling  relationships  between  events,  and  the  parameters  (Buss,  2010). 
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A  state  variable  in  a  DES  model  is  one  that  has  the  possibility  of  changing  value 
at  least  once  during  any  given  simulation  run.  The  number  of  customers  in  the  queue  for 
a  queuing  system  is  an  example  of  a  state  variable.  The  collection  of  all  state  variables  is 
called  the  state  space,  which  gives  a  complete  description  of  the  simulation  model  at  any 
point  in  time  (Buss,  2010). 

Law  (2007)  defines  the  event  as  “instantaneous  occurrence  that  may  change  the 
state  of  the  system.”  The  arrival  of  a  customer  for  a  queuing  system  is  an  example  of  an 
event.  Each  event  is  completely  defined  by  specifying  its  state  transition  function  and  has 
an  associated  event  time  (Buss,  2010). 

Scheduling  relationships  between  events  are  the  rules  that  determine  what  the 
next  event  will  be.  These  relationships  can  be  expressed  as  a  graph  (Buss,  2010). 

The  method  of  time  advance  in  DES  models  is  termed  Next  Event,  which  means 
simulation  time  moves  in  typically  unequal  increments,  jumping  from  the  scheduled  time 
of  one  event  to  another.  The  Next  Event  algorithm  is  shown  in  Figure  3.  (Buss,  2010). 


Figure  3.  Next  Event  Algorithm  [From  (Buss,  2010)] 


Minimum  requirements  for  the  Future  Event  List  (FEE)  in  DES  are:  holding 
pending  events,  keeping  them  in  order,  adding  new  scheduled  events,  and  removing  the 
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next  scheduled  pending  event.  If  canceling  of  events  is  supported,  it  also  needs  to  be  able 
to  find  and  remove  the  cancelled  event  (Buss,  2010).  When  DBS  terminates,  the  PEL  gets 
emptied. 

Parameters  are  the  variables  that  do  not  change  during  a  simulation  run.  The 
maximum  number  of  servers  in  a  queuing  system  is  an  example  of  a  parameter  (Buss, 
2010). 

There  are  several  ways  (terminating  conditions)  in  which  a  DBS  run  can  be 
terminated.  The  simplest  terminating  condition  is  ending  the  simulation  after  a  certain 
amount  of  simulation  time  has  passed.  Another  terminating  condition  is  ending  the 
simulation  after  a  particular  event  has  been  executed  a  preset  number  of  times  (Buss, 
2010). 

Event  graphs  are  an  intuitive  and  powerful  way  to  conceptualize  DBS  models  as 
well  as  a  methodology  for  creating  them.  They  consist  of  nodes  and  directed  edges.  Each 
node  corresponds  to  an  event,  or  state  transition,  and  each  edge  corresponds  to  the 
scheduling  of  other  events.  Each  edge  can  optionally  have  an  associated  Boolean 
condition  and/or  a  time  delay.  The  fundamental  construct  for  event  graphs  is  illustrated  in 
Figure  4.  It  is  interpreted  as  follows:  the  occurrence  of  Event  A  causes  Event  B  to  be 
scheduled  after  a  t  unit  time  delay,  provided  that  condition  (i)  is  true  (Schruben,  1983). 


Figure  4.  Fundamental  Event  Graph  Construct  [From  (Schruben,  1983)] 

There  are  two  more  useful  concepts  in  event  graph  modeling  to  create  simple  and 
flexible  models:  the  ability  to  cancel  scheduled  events,  and  the  ability  to  pass  arguments 
on  scheduling  edges  and  have  these  values  received  by  the  scheduled  event  as  parameters 
(Buss,  2010). 

A  cancelling  edge  is  illustrated  in  Figure  5.  It  is  interpreted  as  follows: 
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Whenever  Event  A  occurs,  then  (following  its  state  transition),  if  condition 
(i)  is  true,  then  only  the  earliest  scheduled  occurrence  of  Event  B  is 
removed  from  the  Event  List.  If  no  such  Event  had  been  previously 
scheduled,  then  nothing  happens.  (Schruben,  1983) 


There  is  no  time  delay  associated  with  a  cancelling  edge. 


Figure  5.  Prototypical  Cancelling  Edge  [From  (Schruben,  1983)] 

The  prototypes  for  a  scheduling  edge  with  arguments  and  an  Event  with 
parameters  is  illustrated  in  Figure  6.  It  is  interpreted  as  follows: 

When  Event  A  occurs,  then  if  condition  (i)  is  true.  Event  B  is  scheduled  to 
occur  (placed  on  the  Event  List)  after  a  delay  of  t,  and  when  it  occurs  its 
parameter  k  will  be  set  to  the  value  of  the  expression  j  at  the  time  it  had 
been  scheduled.  (Buss,  2010) 


Figure  6.  Scheduling  Edge  with  Arguments  and  Events  with  Parameters  [From 

(Schruben,  1983)] 


Finally,  there  is  one  more  useful  concept  in  event  graph  modeling  to  create 
accurate  models  by  breaking  ties  for  events  scheduled  at  exactly  the  same  time:  priorities 
on  scheduling  edges.  This  is  done  by  setting  a  priority  on  the  scheduling  edge.  By 
convention,  higher  numerical  values  represent  higher  priorities.  A  scheduling  edge  with 
priority  p  is  illustrated  in  Figure  7.  (Buss,  2010). 


Figure  7.  Scheduling  Edge  with  Priority  [From  (Schruben,  1983)] 
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It  is  possible  to  build  large  models  using  only  the  basic  event  graph  concepts 
presented  so  far.  But  modular  simulation  components  are  preferred  for  flexibility, 
extensibility  and  scalability  reasons.  We  can  build  large-scale,  complex  models 
effectively  by  creating  small  and  manageable  components  and  connecting  them.  “An 
Event  Graph  component  is  simply  an  Event  Graph  “in  miniature”  —  that  is,  an  object  that 
has  its  own  parameters,  state  variables,  and  events.”  All  components  share  a  common 
Event  List  that  can  keep  track  of  which  event  was  scheduled  by  which  component.  To 
make  a  DES  model  useful  and  interesting,  the  components  do  need  to  have  some  kind  of 
interaction.  That  interaction  is  provided  by  the  SimEventListener  and  Adapter  patterns 
which  are  described  below  (Buss,  2010). 

SimEventListening  logic  is  as  follows: 

One  simulation  component  shows  interest  in  another’s  events  by  explicitly 
being  registered  as  a  SimEventListener  to  it.  If  there  is  a  listener 
relationship  (as  in  Figure  8.  ),  then  whenever  an  Event  from  Source 

occurs,  then  after  it  has  executed  its  state  transitions  and  scheduled  Events, 
the  Event  is  sent  to  Listener.  If  Listener  has  an  Event  that  is  identical  (in 
both  name  and  signature)  to  the  one  it  “hears”  then  it  processes  that  Event 
as  if  it  had  scheduled  it.  The  listening  component  does  not  re-dispatch 
heard  Events  to  its  listeners,  if  it  has  any.”  (Buss,  2010) 


Figure  8.  SimEventListener  Relationship  [From  (Buss,  2010)] 


If  we  desire  an  event  of  one  name  in  a  component  to  cause  another  event  of  a 
different  name  to  occur  in  another  component,  we  use  the  Adapter  pattern.  Unlike  the 
Listener  pattern.  Adapter  works  on  a  single  event  only.  The  Adapter  pattern  seen  in 
Figure  9.  has  this  logic:  Event  A  (source  event)  in  the  Source  component  causes  Event  B 
(adapted  event)  in  the  Listener  component  to  occur  whenever  Event  A  occurs.  The  source 
and  adapted  events  must  have  identical  parameter  lists  for  the  Adapter  to  work  (Buss, 
2010). 
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Figure  9.  Prototype  Adapter  [From  (Buss,  2010)] 


D.  SIMKIT 

Simkit  is  a  free  software  paekage  designed  to  implement  event  graph  models 
easily.  Every  element  in  an  event  graph  model  has  a  eorresponding  element  in  Simkit. 
Eaeh  event  (except  Run)  in  the  event  graph  model  is  implemented  by  a  corresponding 
method  in  a  Simkit  component  (subclass)  that  starts  with  ‘do’  followed  by  the  name  of 
the  event.  Scheduling  edges  are  implemented  by  calling  waitDelayO  method.  The 
following  Simkit  code  is  the  corresponding  implementation  of  the  event  graph  in  Figure 
6.  (Buss,  2010). 

public  void  doA()  { 
intj  =  . . .; 

//  State  transitions  for  Event  A 
if  (i)  { 

waitDelay(“B”,  t,  j); 

} 

} 

public  void  doB(int  k)  { 

//  State  transitions  for  Event  B. 

} 

More  information  about  Simkit  and  event  graph  model  implementation  examples 
in  Simkit  can  be  found  in  (Buss,  2010). 
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III.  RECONNAISSANCE  SQUADRON  WORKFLOW  MODEL 


This  chapter  explains  the  Reeonnaissanee  Squadron  Workflow  model  that  is 
implemented  by  using  the  Simkit  package. 

A.  SCENARIO 

A  Simplified  Reeonnaissanee  Squadron  Workflow  Cycle  is  illustrated  in  Figure 
10.  The  steps  in  this  eyele  are  listed  below: 

•  Reeonnaissanee  requests  for  the  following  day  arrive,  in  bulk,  at  the  squadron 
in  the  late  afternoon. 

•  Flight  planning  for  the  next  day  oecurs.  This  entails  creating  aireraft,  pilot, 
camera  and  flight  time  temporary  assignments  by  considering  the 
reeonnaissanee  request  attributes  such  as  priority,  due  date,  image  type  and 
angle  type. 

•  Flight  execution  for  the  speeific  reeonnaissanee  missions  at  take-off  times. 
There  are  a  few  factors  that  affect  the  sueeess  of  the  flight  exeeution,  sueh  as 
aircraft/camera  defeet(s),  bad  weather  eonditions,  and  pilot  filming  error. 
Except  for  aireraft  defeets,  all  of  the  other  factors  are  determined  when  the 
evaluation  of  the  film  is  started. 

•  Assessment  (processing  and  interpretation)  of  films  after  aircraft  landing  and 
then  report  generation  based  on  the  reeonnaissanee  request  requirements. 
Generated  reports  are  sent  to  the  intelligenee  users. 
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Figure  10.  Simplified  Reconnaissance  Squadron  Workflow  Cycle 


Table  1  shows  four  different  scenarios  in  Reconnaissance  Squadron  Workflow 
Cycle  based  on  situations  and  configurations.  There  is  a  specific  model  for  each  scenario. 


Situation 

Configuration 

Type  1 

Type  2 

War 

RF-4 

F-16 

Peace 

RF-4 

F-16 

Table  1 .  Situation/Configuration  Table  for  Different  Scenarios 


Figure  11.  shows  the  flight  planning  for  a  peace-time  situation  with  RF-4 
configuration.  The  notional  Air  Base  is  located  at  the  (0,  0)  coordinate.  The  notional 
target  zone  has  (-65,  20)  as  the  upper  left  coordinate  and  (-35,-50)  as  the  lower  right 
coordinate.  Each  coordinate  in  this  target  zone  represents  a  target.  Each  reconnaissance 
request  has  a  specific  target.  In  this  peace  situation,  reconnaissance  requests  that  have 
adjacent  targets  are  combined  into  one  mission  if  their  camera  requirements  are  the  same, 
which  means  requiring  the  same  image  and  angle  type.  Combining  adjacent  targets 
provides  more  training  time  for  pilots  and  efficient  usage  of  resources. 
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Figure  12.  shows  the  reconnaissance  planning  for  war  situation.  RF-4  aircraft 
missions  include  only  one  reconnaissance  request  (that  is,  no  target  combining  occurs). 
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Figure  12.  Flight  Leg  Planning  for  RF-4,  War  Situation  Reconnaissance  Missions 
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Figure  13  shows  the  reconnaissance  mission  planning  for  peace  situation.  In  a 
peace  situation,  reconnaissance  requests  that  have  adjacent  targets  are  combined  into  one 
mission  if  their  camera  requirements  are  the  same,  which  means  requiring  the  same 
image  and  angle  type.  F-16  aircrafts  carry  an  EO/IR  pod,  which  can  take  both  EO  and  IR 
imagery  simultaneously.  Besides,  when  the  aircraft  is  in  the  line  of  sight  with  the  antenna 
and  within  its  uplink/downlink  parameter,  new  missions  can  be  uploaded  or  imagery  can 
be  downloaded.  In  this  study,  only  the  downlink  capability  is  modeled. 
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Figure  13.  Flight  Leg  Planning  for  F-16  Reconnaissance  Missions 


B.  ASSUMPTIONS 

Common  assumptions  used  in  all  models  are  listed  below: 

•  Bulk  reconnaissance  requests  arrive  to  the  squadron  every  day  at  17:00  for 
the  following  day’s  flight  planning. 

•  The  duration  of  a  flight  is  calculated  by  using  a  heuristic  algorithm 
(calculateFlightDuration  method  of  FlightsPlanner  class)  that  is  based  on  a 
standard  flight  pattern;  this  is  explained  in  the  event  graphs  section. 

•  On  average,  aircrafts  cover  one  mile  in  0.3  minutes. 
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•  There  is  a  fifteen  minutes  gap  between  consecutive  flights. 

•  After  entering  the  target  area  for  each  target,  it  takes  ten  minutes  of  flight 
time  to  capture  the  images  of  the  target. 

•  Even  though  both  day  (requires  optic  imagery)  and  night  (requires  infrared 
imagery)  reconnaissance  missions  are  conducted  in  specific,  limited  times, 
there  is  no  specific  or  limited  time  for  the  evaluation  of  the  films/imagery. 
Image  analysts  interpret  and  evaluate  films/imagery  when  they  arrive  and  they 
do  this  job  until  all  of  the  imagery  in  the  queue  is  evaluated. 

Scenario  specific  assumptions  are  listed  separately  in  order  to  highlight  the 
differences  between  the  four  models. 

1.  Configuration  of  the  RF-4 

•  RF-4  aircraft  flights  require  two  pilots  for  reconnaissance  missions. 

•  For  the  evaluation  of  the  films,  two  image  analysts  are  assigned  for  each 
target. 

•  Only  one  sensor  (camera)  is  mounted  to  the  RF-4  aircraft  for  the 
reconnaissance  mission.  There  is  no  spare  sensor  planning. 

•  All  requests  require  new  reconnaissance  missions.  There  is  no  usage  of 
responding  requests  from  existing  image  library. 

a.  Situation:  Peace 

•  Reconnaissance  missions  requiring  optic  imagery  can  only  be 
conducted  between  10:00  and  16:00. 

•  Reconnaissance  missions  requiring  infrared  (IR)  imagery  can  only 
be  conducted  between  18:00  and  24:00. 
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•  Main  flight  planning  occurs  at  03:00  each  day.  Any  other  required, 
additional  flight  planning  occurs  during  flight  executions  after  specific 
events  such  as  aircraft  defect(s),  camera  defect(s)  or  pilot  filming 
error. 

•  During  flight  planning,  reconnaissance  requests  that  have  adjacent 
targets  are  combined  into  one  mission  if  their  camera  requirements  are 
the  same,  which  means  requiring  the  same  image  and  angle  type  (see 
Figure  11.). 

•  Pilots  are  required  to  take  two  hours  of  break  (trestingiimeForPiiots)  after 
successful  flights,  due  to  crew  rest  policies. 

•  After  an  RF-4  aircraft  lands,  it  takes  ten  minutes  to  take  the  aircraft 
to  the  hangar  and  stop  the  engines.  If  aircraft  lands  without  defect(s),  it 
goes  to  seventy-five  minutes  routine  maintenance  (turnaround  time). 

•  Due  dates  are  assigned  according  to  the  priorities  of  the 
reconnaissance  requests.  The  highest  priority  reconnaissance  requests 
have  the  earliest  due  dates. 

b.  Situation:  War 

•  Reconnaissance  missions  requiring  optic  imagery  can  only  be 
conducted  between  10:00  and  17:00,  which  is  one  hour  more  than  in 
the  peace  situation. 

•  Reconnaissance  missions  requiring  infrared  (IR)  imagery  can  only 
be  conducted  between  19:00  and  02:00.  The  night  mission  hours  are 
shifted  one  hour  from  the  peace  situation. 

•  Main  flight  planning  occurs  at  05:00  each  day.  Any  other  required, 
additional  flight  planning  occurs  during  flight  executions  after  specific 
events  such  as  aircraft  defect(s),  aircraft  interception,  camera  defect(s) 
or  pilot  filming  error. 
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•  In  contrast  to  the  peace  situation,  reconnaissanee  requests  that  have 
adjacent  targets  are  not  eombined  into  one  mission  (see  Figure  12.  ). 

•  War  situation  requires  extreme  measures,  so  that  pilots  ean  be 
assigned  to  eonsecutive  flights  without  any  rest  time. 

•  After  an  RF-4  aireraft  lands,  it  takes  ten  minutes  to  take  the  aireraft 
to  the  hangar  and  stop  the  engines.  If  the  aircraft  lands  without 
defect(s),  it  goes  to  one  hour  routine  maintenance  (turnaround  time). 

•  All  reconnaissance  requests  have  the  same  due  date,  whieh  is 
creation  time  plus  one  day.  In  war  situations,  due  dates  are  more 
restrietive  than  peaee  situations. 

•  When  the  aircraft  is  intercepted,  all  of  the  resourees  (camera, 
aireraft  and  pilots)  are  assumed  to  be  lost. 

Configuration  of  the  F-16 

•  The  F-16  aircraft  is  modeled  as  a  single  seat,  that  is,  it  requires  only  one 
pilot. 

•  For  the  evaluation  of  images,  one  image  analyst  is  assigned  for  eaeh 
target. 

•  Only  one  sensor  (camera)  is  mounted  to  the  F-16  aircraft  pod.  This  pod  is 
dual  band,  that  is,  it  can  take  both  IR  and  Optie  images  simultaneously.  There 
is  no  spare  sensor  planning. 

•  During  flight  planning,  reconnaissance  requests  that  have  adjacent  targets 
are  eombined  into  one  mission  if  their  angle  types  are  same  (see  Figure  13). 

a.  Situation:  Peace 

•  Reeonnaissanee  missions  requiring  optic  imagery  ean  only  be 
conducted  between  10:00  and  16:00.  Also,  IR  imagery  requests  ean  be 
planned  during  these  hours. 


•  Reconnaissance  missions  for  night  are  only  possible  with  IR 
imagery,  and  can  only  be  conducted  between  18:00  and  24:00. 

•  Main  flight  planning  occurs  at  03:00  each  day.  Any  other  required, 
additional  flight  planning  occurs  during  flight  executions  after  specific 
events  such  as  aircraft  defect(s),  camera  defect(s)  or  pilot  filming 
error. 

•  Pilots  are  required  to  take  two  hours  of  break  (trestingTimeForPiiots)  after 
successful  flights,  due  to  crew  rest  policies. 

•  After  an  F-16  aircraft  lands,  it  takes  ten  minutes  to  take  the  aircraft 
to  the  hangar  and  stop  the  engines.  If  an  aircraft  lands  without 
defect(s),  it  goes  to  one  hour  routine  maintenance  (turnaround  time). 

•  Due  dates  are  assigned  according  to  the  priorities  of  the 
reconnaissance  requests.  Highest  priority  reconnaissance  requests  have 
the  earliest  due  dates. 

•  If  so  specified,  reconnaissance  requests  can  be  responded  to  using 
information  from  old  missions,  that  is,  from  an  imagery  archive,  if  the 
previous  mission  is  not  older  than  one  week.  If  recent  images  are  not 
available,  then  a  flight  mission  is  planned. 

Situation:  War 

•  Reconnaissance  missions  requiring  optic  imagery  can  only  be 
conducted  between  10:00  and  17:00. 

•  Reconnaissance  missions  requiring  IR  imagery  can  only  be 
conducted  between  19:00  and  02:00. 

•  Main  flight  planning  occurs  at  05:00  each  day.  Any  other  required, 
additional  flight  planning  occurs  during  flight  executions  after  specific 
events  such  as  aircraft  defect(s),  aircraft  interception,  or  pod  defect(s). 

•  Pilots  can  be  assigned  to  consecutive  flights  without  any  rest  time. 
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•  After  an  F-16  aircraft  lands,  it  takes  ten  minutes  to  take  the  aircraft 
to  the  hangar  and  stop  the  engines.  If  the  aircraft  lands  without 
defect(s),  it  goes  to  one  hour  routine  maintenance  (turnaround  time). 

•  All  reconnaissance  requests  have  the  same  due  date,  which  is  the 
creation  time  plus  one  day. 

•  When  the  aircraft  is  intercepted,  all  of  the  resources  (pod,  aircraft 
and  pilot)  are  assumed  to  be  lost. 

C.  LIMITATIONS,  CONSTRAINTS  AND  REQUIREMENTS 

In  the  model,  bulk  reconnaissance  requests  arrive  to  the  squadron  every  day  at 
17:00.  The  model  does  not  consider  the  exact  creation  time  of  each  reconnaissance 
request  or  the  creators  of  the  reconnaissance  request  who  affect  the  priority  or  report  type 
of  the  reconnaissance  request. 

After  report  writing  is  completed,  it  is  assumed  that  the  mission  is  successfully 
finished.  However,  in  reality,  there  are  a  few  more  steps  in  this  process  such  as 
transmission  and  evaluation  of  the  reports.  In  this  thesis,  only  the  steps  through  the  report 
writing  are  modeled. 

The  RF-4  aircraft  can  carry  more  than  one  sensor  in  its  specially-designed  nose 
compartment.  In  addition  to  these  sensors,  it  can  carry  a  sensor  pod  as  well.  Since  a 
successful  reconnaissance  mission  requires  only  one  reliable  sensor,  the  peace  and  war 
situation  RF-4  models  assume  that  there  is  only  one  sensor  mounted  on  the  aircraft.  In 
reality,  spare  sensors  are  also  mounted  on  the  aircraft  in  order  to  increase  the  probability 
of  mission  success. 

There  are  also  some  restrictions  about  the  operational  usage  of  the  sensors,  such 
as  the  minimum  operational  altitude  or  ground  speed.  However,  the  model  does  not  take 
those  limitations  into  account. 
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The  model  does  not  also  take  into  account  the  different  types  of  sensors,  nor  the 
logistic  needs  for  sensors  and  aircrafts.  While  making  experiments  on  the  model,  defect 
probabilities  and  maintenance  times  for  aircrafts  and  sensors  are  assigned  based  on 
authors’  experience. 

All  targets  are  assumed  to  be  fixed  targets;  that  is,  they  are  not  mobile  and  their 
coordinates  are  fixed.  This  allows  for  planning  in  advance  for  the  next  day.  There  is  no 
reconnaissance  request  during  the  flight  hours  for  emerging  targets. 

In  this  study,  different  scenarios  and  configurations  are  modeled  to  address  the 
purpose  of  this  study  -  namely,  to  find  out  what  important  factors  are  affecting  the  two 
different  reconnaissance  systems.  Comparing  the  two  different  reconnaissance  systems  is 
not  the  purpose  of  this  study. 

D.  EVENT  GRAPHS 

The  event  graphs  in  the  figures  below  do  not  indicate  state  transitions;  these  are 
discussed  in  the  paragraphs  above  the  figures. 

1.  Configuration  of  the  RF-4 
a.  Situation:  Peace 

Figure  14  shows  the  main  components  of  the  RF-4  configuration  in  a 
peace  situation  model.  There  are  five  main  components  in  the  model: 
RecSquadronWorkflow,  RecRequestsCreator,  FlightsPlanner,  FlightsExecutor  and 
ReportsGenerator.  These  main  components  are  independent  of  each  other,  but  they  are 
connected  by  using  listener  and  adapter  patterns  to  make  the  whole  model  run  accurately. 
Each  main  component  is  explained  below  in  detail. 
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Figure  14.  Conf.:  RF-4  /  Sit.:  Peace  -  Components  with  Listener  and  Adapter  Patterns 

Figure  15  depicts  the  event  graph  of  the  RecSquadronWorkflow  class.  The 
RecSquadronWorkflow  class  works  as  a  simulation  clock.  It  triggers  the  creation  of  bulk 
reconnaissance  requests  and  main  flight  planning  every  day  at  the  same  hour  during  the 
simulation  run.  Simulation  starts  at  00:00  with  the  Run  event.  This  schedules  a 
CreateRecRequests  event  with  a  trecReqCreiime  time  delay  (which  corresponds  to  17:00), 
and  PlanFlights  event  with  tpianfiightsiime  time  delay  (which  corresponds  to  03:00  the  next 

day).  The  delay  values  are  in  minutes,  e.g.,  =  17  hours.  Both  CreateRecRequests 

and  PlanFlights  events  schedule  themselves  to  occur  every  day  at  the  same  time  by  using 
tminutesinADay  delay  which  cquals  twenty-four  hours. 
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Parameters 

tnumberMinutesinAOay  =  Constant  time  foF  daily  scheduling  of  events 
tracF^c-sTime  =  Constant  time  for  daily  rec.  req.  creation 
tpiaiRightsTime  =  Constant  time  for  daily  main  flight  planning 

Figure  15.  Conf.:  RF-4  /  Sit.:  Peace  -  Reconnaissance  Squadron  Workflow  Event  Graph 


Figure  16  shows  the  event  graph  of  the  RecRequestsCreator  component. 
The  RecRequestsCreator  component  deals  with  the  creation  of  daily  bulk  reconnaissance 
requests,  recording  of  successful  reconnaissance  request  mission  targets  to  the  target 
archive,  and  removal  of  unresponded  reconnaissance  request  targets  from  the  target 
archive. 

The  CreateRecRequestsO  event  generates  a  rounded  random  number  (n) 
based  on  a  triangular  distribution  whose  parameters  are  set  to  the  input  taken  from  the 
user.  Then  it  passes  zero  and  this  random  number  to  the  CreateRecRequest(i,m)  event  as 
parameters.  The  CreateRecRequest(i,m)  event  creates  reconnaissance  request  (r)  by 
calling  the  createNewRecRequest()  method,  schedules  an  Arrival  event  with  created 
reconnaissance  request  (r),  and  schedules  itself  m  times  with  increasing  i.  The 
createNewRecRequestO  method  creates  an  instance  of  reconnaissance  request  class, 
RecRequest,  which  includes  fields  such  as  priority  index,  due  date,  camera  type  and 
target  location.  The  RemovefromArchive  event  allows  the  allocation  of  cancelled 
reconnaissance  request  (r)  targets  to  newly-created  reconnaissance  requests. 
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Figure  16.  Conf.:  RF-4  /  Sit.:  Peace  -  Reconnaissance  Requests  Creator  Event  Graph 

Figure  17  shows  the  event  graph  of  the  FlightsPlanner  component.  The 
FlightsPlanner  component  plans  reconnaissance  flights  dynamically  based  on  availability 
of  resources  such  as  aircraft,  pilots,  and  cameras.  During  flight  planning  it  combines 
reconnaissance  requests  that  have  adjacent  targets  into  one  flight  and  calculates  flight 
durations.  FlightsPlanner  keeps  track  of: 

•  Aircraft,  pilot,  and  camera  inventories. 

•  Planned  flights  list. 

•  Two  separate  queues  for  daylight  and  night  reconnaissance  requests. 

In  the  RecRequestArrival(r)  event,  the  arrival  time  of  a  reconnaissance 
request  is  stamped  and  then  this  reconnaissance  request  (r)  is  added  to  the  relevant  queue. 
The  numberRecRequestArrivals  state  variable  is  incremented  by  one.  Then  the 
RecRequestArrival(r)  event  schedules  the  RemoveRecRequest(r)  method  with  a 
tdueDate  -  simTime  delay.  Each  rcconnaissance  request  should  be  responded  to  before  its  due 
date  passes.  The  RemoveRecRequest  event  removes  reconnaissance  requests  from  the 
relevant  queue  and  increments  the  number RemovedRecRequests  state  variable  by  one. 

The  RecReportProvided  event  first  updates  the  totalTimeAtWorkFlow 
state  variable.  Then  it  cancels  the  RemoveRecRequest  event  for  relevant  reconnaissance 
requests. 

The  PlanFlights  event  makes  the  main  flight  planning  based  on  availability 
of  cameras,  aircraft,  and  pilots  within  daily  flight  hours  for  both  day  and  night  flights. 
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During  flight  planning,  it  combines  reconnaissance  requests  that  have  adjacent  targets 
into  one  flight  and  calculates  flight  durations.  The  PlanFlights  event  schedules  StartFlight 
events  for  planned  flights  (f)  with  relevant  tpiannedPiightrime  -  simiime  delays. 


The  duration  of  a  flight  is  calculated  by  using  a  heuristic  algorithm  in  the 
calculateFlightDuration  method.  In  this  algorithm,  it  is  assumed  that  the  aircraft  flies 
from  base  to  targets  which  are  sorted  in  ascending  order  according  to  their 
reconnaissance  request  due  dates.  The  algorithm  first  initializes  the  flight  duration  to 
zero.  Then  it  calculates  the  total  distance  traveled  during  a  flight  by  adding  distances 
between  flight  points.  The  distance  between  two  flight  points  is  calculated  by  using  a 
variant  of  the  Pythagorean  Theorem  (Purple  Math,  2010).  The  total  distance  is  multiplied 
by  the  aver ageTimePer Mile  constant  and  added  to  the  flight  duration.  The  number  of 
targets  in  the  flight  mission  is  multiplied  by  the  averageTimeForTarget  constant  and  the 
result  is  also  added  to  the  flight  duration. 

The  UpdateFlightsForFCDefect  event  first  cancels  the  flights  (f)  which 
cannot  be  completed  at  their  scheduled  time  because  of  camera  defect(s)  and  then  tries  to 
plan  new  flights  for  those  canceled  ones  by  using  the  planNewFlight  method. 

The  UpdateFlightsForACDefect  event  first  cancels  the  flights  (f)  which 
cannot  be  completed  at  their  scheduled  time  because  of  aircraft  defect(s)  and  then  tries  to 
plan  new  flights  for  those  canceled  ones  by  using  the  planNewFlight  method. 

The  PlanNewFlight  event  tries  to  plan  a  new  flight  for  its  argument  flight 
by  using  the  planNewFlight  method.  It  also  adds  the  reconnaissance  requests  (r)  of  the 
cancelled  or  unsuccessful  flight  to  the  relevant  queue. 

The  planNewFlight  private  method  tries  to  plan  a  new  flight  (nF)  for  a 
cancelled  or  unsuccessful  flight  by  using  available  resources  (cameras,  aircrafts  and 
pilots).  If  a  new  flight  can  be  planned  it  schedules  the  StartFlight  event  with 
tpiannedPiightrime  -  simTime  delay  and  Updates  the  flights  list. 

The  JoinFlightCamerasPool  event  updates  cameras’  availability  times. 

The  JoinPilotsPool  event  updates  pilots’  availability  times. 
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The  JoinAirCraftsPool  event  updates  availability  time  for  the  aircraft. 

The  Updateinventories  event  updates  inventories  of  cameras,  pilots,  and 
aircraft  after  an  aircraft  takes  off  for  a  reconnaissance  mission. 


The  RemoveFlightAndRecRequests  event  updates  the  flights  list  and 
relevant  reconnaissance  request  queue  after  an  aircraft  takes  off  for  a  reconnaissance 
mission. 


^dueoato-amrnn  ~  i^^quest  flightList  =  List  of  planned  flights 

~  ^P  Planned  flight  daylightRecRequestsQueue  =  Queue  for  day  flight  requiring  requests 
time  of  rec.  request  nightRecRequestsQueue  =  Queue  for  night  flight  requiring  requests 

numberRecRequestArrivais  =  Number  of  arrived  rec.  requests 
numberRemovedRecRequests  =  Number  of  non-responded  rec.  requests 
totalTimeAtWorkFlow  =  Duration  between  rec.  request  arrival  and  its 

report  generation 

Figure  17.  Conf :  RF-4  /  Sit.:  Peace  -  Flights  Planner  Event  Graph 


Figure  18  shows  the  event  graph  of  the  FlightsExecutor  component.  The 
FlightsExecutor  component  deals  with  reconnaissance  flights  after  flight  operations  and 
most  of  the  state  variable  changes  in  the  whole  model. 

First,  the  TakeOff  event  schedules  the  UpdateQueueAndList  event  with 
the  relevant  flight  (f)  and  resource  group  (rG)  parameters. 
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Second,  the  TakeOff  event  schedules  the  AssignResources  event  with 
resource  group  (rG)  parameter.  Unless  the  simulation  time  is  less  than  the  availability 
times  of  any  resource  group  component  (camera,  aircraft  and  pilots),  the  TakeOff  event 
decrements  the  relevant  numberAvailableFlightCameras  and  numberAvailableAirCrafts 
state  variables  by  one  and  decrements  the  numberAvailablePilots  state  variable  by  two. 

Third,  the  TakeOff  event  generates  a  random  number  for  aircraft  defect 
probability  (p).  If  this  number  is  less  than  or  equal  to  the  user  input  for  aircraft  defect 
probability,  the  TakeOff  event  schedules  LandForACDefect  by  passing  relevant  flight  (f) 
with  taircraftDefectRecogTime  delay.  If  p  is  greater  than  the  user  input  for  aircraft  defect 
probability,  then  the  TakeOff  event  schedules  the  Land  event  by  passing  relevant  flight 

(f)  with  tfiightDuration  delay. 

Finally,  if  the  simulation  time  is  less  than  the  availability  times  of  any 
resource  group  component  (camera,  aircraft,  and  pilots)  the  TakeOff  event  schedules  the 
FlightCancelled  event  by  passing  relevant  flight  (f). 

The  Land  event  schedules  three  events.  First,  it  schedules  the 
StartMissionEvaluation  event  by  passing  relevant  flight  (f)  with  a  taverageEnginestopTime  delay, 
which  is  the  time  between  aircraft  landing  and  turning  off  the  aircraft  engines.  Films  can 
be  removed  after  the  aircraft  engines  stop.  Second,  it  schedules  the  StartACMaintenance 
event  by  passing  relevant  aircraft  (a)  with  a  taverageEnginestopTime  delay.  Third,  it  schedules 
the  NewAvailablePilots  event  by  passing  relevant  pilots  (p)  with  a  trestingiimeForPiiots  delay. 

The  LandForACDefect  event  schedules  four  events.  First,  it  schedules  the 
NewAvailableFlighCamera  event  by  passing  relevant  camera  (fc)  with  a 
taverageEnginestopTime  delay.  Sccond,  it  schcdulcs  the  NcwAvailablePilots  event  by  passing 
relevant  pilots  (p).  Third,  it  schedules  the  StartACDefectRepair  event  by  passing  relevant 
aircraft  (a)  with  a  taverageEnginestopTime  delay.  Fourth,  it  schedules  the  FlightCancelled  event 
by  passing  relevant  flight  (f)  with  a  taverageEnginestopTime  delay. 

First,  the  StartACDefectRepair  event  updates  aircraft  inventory.  Second,  it 
schedules  the  ACDefectEffect  event.  Finally,  it  schedules  the  EndACDefectRepair  event 
by  passing  relevant  aircraft  (a)  with  a  tairCraftoefRepTime  delay. 
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The  EndACDefectRepair  event  schedules  NewAvailableAirCraft  event  by 
passing  relevant  aircraft  (a).  The  StartACMaintenance  event  updates  aircraft  inventory. 
Then  it  schedules  the  EndACMaintenance  event  by  passing  relevant  aircraft  (a)  with  a 
tavgMaintTimeForAirCrafts  delay.  The  EndACMaintcnancc  event  schedules  the 
NewAvailableAirCraft  event  by  passing  relevant  aircraft  (a). 


The  NewAvailableFlightCamera  event  first  increments  the  relevant 
numberAvailableFlightCameras  state  variable  by  one.  The  NewAvailableAirCraft  event 
increments  the  numberAvailableAirCrafts  state  variable  by  one.  The  NewAvailablePilots 
event  increments  the  numberAvailablePilots  state  variable  by  two. 


Parameters  State  Variables 

totalNumberFlightCameras  Q  =  Number  of  flight  cameras  numberAvailableFlightCameras  =  Number  of  available  flight  cameras 
totalNumberAirCrafts  =  Number  of  aircrafts  numberAvailableAirCrafts  =  Number  of  available  aircrafts 

totalNumberPilots  =  Number  of  pilots  numberAvailablePilots  =  Number  of  available  pilots 

W»Dt«ion-F»Ohttime 

WnMMMRMoann*  "  Recognition  of  aircraft  defect  time 
^EfigsiapTknt  =  Average  time  for  engine  shut  off 

UTiMiortw.-  “ms  fof  pilot* 

Ucrieiorf^TVm^  Aircraft  defect  repair  time 

Average  time  for  aircraft  maintenance 

Figure  18.  Conf.:  RF-4  /  Sit.:  Peace  -  Flights  Executor  Event  Graph 
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Figure  19  depicts  the  event  graph  of  ReportsGenerator  component.  The 
ReportsGenerator  component  deals  with  the  operations  after  a  successful  reconnaissance 
flight,  that  is,  interpreting  the  films  and  writing  reconnaissance  reports  based  on  their 
requirements. 

The  InterpretFCFilms  event  can  schedule  four  different  events.  First,  it 
generates  a  random  number  for  camera  defect  probability  (pi).  If  this  number  is  less  than 
or  equal  to  the  user  input  for  the  relevant  camera  defect  probability,  it  schedules  the 
StartFCDefectRepair  event  by  passing  relevant  camera  (fc).  Second,  if  pi  is  greater  than 
the  user  input  for  relevant  camera  defect  probability,  then  the  InterpretFCFilms  event 
schedules  the  NewAvailableFlightCamera  event  by  passing  the  relevant  camera  (fc). 

Third,  based  on  generated  random  numbers,  if  there  is  a  camera  defect  or 
pilot  filming  error  or  bad  weather  condition,  then  the  InterpretFCFilms  event  schedules 
the  BadFilmResults  event  by  passing  relevant  flight  (f).  Fourth,  based  on  generated 
random  numbers,  if  there  is  no  camera  defect,  no  pilot  filming  error,  and  no  bad  weather 
condition,  then  the  InterpretFCFilms  event  generates  random  report  writing  times  for 
reconnaissance  requests  in  the  flight  (f)  and  adds  these  reconnaissance  requests  to  the 
reconnaissance  requests  queue.  If  there  is  an  available  image  analyst,  the 
InterpretFCFilms  event  schedules  the  StartReportWriting  event. 

First,  the  StartFCDefectRepair  event  updates  the  camera  inventory. 
Second,  it  schedules  the  FCDeffectEffect  event  by  passing  relevant  camera  (fc).  Finally, 
the  StartFCDefectRepair  event  schedules  the  EndFCDefectRepair  event  by  passing 
relevant  camera  (fc)  with  a  tfiightcameraRepairTime  delay.  The  EndFCDefectRepair  event 
schedules  the  NewAvailableFlightCamera  event  by  passing  relevant  camera  (fc). 

The  StartReportWriting  event  takes  the  first  reconnaissance  request  (r)  in 
the  queue,  removes  it  from  the  queue  and  schedules  the  EndReportWriting  event  by 
passing  that  reconnaissance  request  (r)  with  a  treportwritingiime  delay.  The 
StartReportWriting  event  also  decrements  the  number  Available  Analysts  state  variable  by 
two. 
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The  EndReportWriting  event  increments  the  numberProvidedRecRequests 
state  variable  by  one  and  the  number  Available  Analysts  state  variable  by  two.  If  there  is 
any  reconnaissance  request  in  the  queue  waiting  for  the  report  writing  process,  the 
EndReportWriting  event  schedules  the  StartReportWriting  event. 


Parameters 

totalNumberAnalysts  =  Number  of  analysts 
ffcRspwrims  =  Camera  repair  time 
WortWrtungnn.  =  Repoft  Writing  time 


State  Variables 

numberAvaiiableAnalysts=  Number  of  avaiiabie  anaiysts 
numberProvidedRecRequests  =  Number  of  provided  requests 
recRequestsQueue  =  Queue  for  rec.  request 


Figure  19.  Conf :  RF-4  /  Sit.:  Peace  -  Reconnaissance  Request  Report  Generator  Event 

Graph 


b.  Situation:  War 

Figure  20  shows  the  main  components  of  the  RF-4  configuration  in  a  war 
situation  model.  As  in  the  peace  situation,  there  are  five  main  components  in  the  model: 
the  RecSquadronWorkflow,  RecRequestCreator,  FlightsPlanner,  FlightsExecutor  and 
ReportsGenerator.  Even  though,  during  the  peace  time,  training  is  done  according  to 
“Train  like  you  fighf  ’  principle,  there  are  significant  changes  in  the  war  situation.  Events 
in  red  are  either  new  or  changed  events  specific  to  the  war  scenario.  The  rest  of  the  events 
are  the  same  as  in  the  peace  scenario.  Each  main  component  is  explained  below  in  detail. 
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Figure  20.  Conf.:  RF-4  /  Sit.:  War  -  Components  with  Listener  and  Adapter  Patterns 

The  RecSquadronWorkflow  eomponent  of  the  RF-4  configuration  war 
situation  scenario  is  same  as  the  RF-4  configuration  peace  situation  scenario,  except  that 
the  value  for  tpianiiightsTime  time  delay  is  incremented  by  two  hours  to  allow  for  more  flight 
hours. 

The  RecRequestsCreator  component  of  RF-4  configuration  war  situation 
scenario  is  same  as  RF-4  configuration  peace  situation  scenario.  Only  the  due  dates  given 
to  reconnaissance  requests  are  different.  In  the  peace  scenario,  the  duration  between 
arrival  and  due  date  of  each  reconnaissance  request  is  random.  In  the  war  scenario,  this 
duration  is  just  one  day. 

Figure  21  shows  the  event  graph  of  the  FlightsPlanner  component.  Events 
in  red  are  either  new  or  changed  events  specific  to  the  war  scenario.  The  rest  of  the  events 
are  the  same  as  in  the  peace  scenario,  except  that  the  adjacent  target  combination  possible 
in  the  peace  scenario  is  not  done  in  the  war  scenario. 
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The  UpdateFlightsForFCEffect  event  first  caneels  the  flights  which  cannot 
be  completed  at  their  scheduled  times  because  of  either  camera  defect(s)  or  camera  loss 
(due  to  aircraft  interception)  and  tries  to  plan  new  flights  for  those  canceled  ones  by  using 
the  planNewFlight  method. 

The  UpdateFlightsForACEffect  event  first  cancels  the  flights  which 
cannot  be  completed  at  their  scheduled  times  because  of  either  aircraft  defect(s)  or 
aircraft  loss  (due  to  aircraft  interception)  and  tries  to  plan  new  flights  for  those  canceled 
ones  by  using  the  planNewFlight  method. 

The  UpdateFlightsForPilotsEffect  event  first  cancels  the  flights  which 
cannot  be  completed  at  their  scheduled  times  because  of  pilot  loss  (due  to  aircraft 
interception),  and  tries  to  plan  new  flights  for  those  canceled  ones  by  using  the 
planNewFlight  method. 

The  RemoveFromInventories  event  updates  inventories  of  cameras,  pilots, 
and  aircraft  after  an  aircraft  is  intercepted.  Also,  if  inventories  of  flight  cameras  and 
aircrafts  drop  to  0,  or  the  inventory  of  pilots  drops  to  1,  then  the  simulation  stops.  This 
means  that  the  reconnaissance  squadron  cannot  accomplish  its  duties  because  either  there 
are  no  personnel  or  equipment. 

The  RemoveFlightAndRecRequest  event  updates  the  flights  list  and 
relevant  reconnaissance  request  queue  after  an  aircraft  takes  off  for  a  reconnaissance 
mission. 
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^duaoato-HtaiTiiiie  =  up  to  due  date  of  rec.  request  flightList  =  List  of  planned  flights 

Viam0(«^tnii»^Tinie  =  up  to  Planned  flight  daylightRecRequestsQueue  =  Queue  for  day  flight  requiring  requests 
time  of  rec.  request  nightRecRequestsQueue  =  Queue  for  night  flight  requiring  requests 

numberRecRequestArrlvals  =  Number  of  arrived  rec.  requests 
numberRemovedRecRequests  =  Number  of  non-responded  rec.  requests 
totalTimeAtWorkFlow  =  Duration  between  rec.  request  arrival  and  its 

report  generation 


Figure  21.  Conf.:  RF-4  /  Sit.:  War  -  Flights  Planner  Event  Graph 


Figure  22  shows  the  event  graph  of  the  FlightsExeeutor  component. 
Events  in  red  are  either  new  or  changed  events  specific  to  the  war  scenario.  The  rest  of 
the  events  are  the  same  as  in  the  peace  scenario,  except  the  non-zero  probability  of 
reconnaissance  aircraft  interception  by  the  enemy  affects  the  entire  reconnaissance  cycle 
due  to  lost  resources  such  as  pilots,  aircraft,  and  cameras. 

The  TakeOff  event  generates  a  random  number  for  aircraft  defect 
probability  (p).  If  p  is  greater  than  the  user  input  for  aircraft  defect  probability,  then  the 
TakeOff  event  schedules  the  ArriveToTarget  event  by  passing  relevant  flight  (!)  as  a 
parameter  with  tfiightDuration/2  delay.  If  simulation  time  is  less  than  the  availability  time  of 
any  resource  group  (rG)  component  (camera,  aircraft  and  pilots),  the  TakeOff  event 
schedules  the  NewFlightRequired  event  by  passing  relevant  flight  (f). 
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A  random  number  for  aircraft  intercept  probability  (p)  is  generated  and 
compared  with  the  user  input  for  aircraft  intercept  probability.  If  the  aircraft  intercept 
probability  (p)  is  less  than  or  equal  to  user  threshold,  then  the  ArriveToTarget  event 
schedules  the  ACIntercepted  event.  If  not,  the  ArriveToTarget  event  schedules  the  Land 
event  with  tfiightDuration/2  delay. 


The  ACIntercepted  event  schedules  five  events.  First,  it  schedules  the 
ResourceLost  event  by  sending  the  pertinent  resource  group  (rG).  Second,  it  schedules 
the  NewFlightRequired  event  by  sending  relevant  flight  (f).  Third,  it  also  schedules  the 
ACEffect  event  by  sending  pertinent  aircraft  (a).  Fourth,  it  also  schedules  the  FCEffect 
event  by  sending  pertinent  flight  camera  (fc).  Finally,  the  PilotsEffect  event  is  scheduled 
by  sending  relevant  pilots  (p). 


totalNumberFllghtCameras  Q  -  Number  of  fight  cameras  numberAvallableFllghICameras  ~  Number  of  available  flight  cameras 

luUilNuriibiirAirCrHfU»  ■  Nuinber  uf  ufrurfeina  riurTiUNAvdlHblnAirCrana  ■  Number  uF  evelbble  tdrurefle 

totalNumberPllota  ■  Number  of  pllola  numberAvallablePllota  ■  Number  of  available  pilots 

WiDwaion^  ■  Flight  time  until  arrival  to  the  target/land 

UMiKMenMocniM  =  F^cogn  tton  of  aircraft  defect  time 

UgEi^tepTimi  =  Averags  time  for  engine  shut  aff 

UoiiiMRvTkr*"  Aircraft  defect  repair  time 

^m^tTkmwkcimr  Average  time  for  aircraft  maintenance 

Figure  22.  Conf.:  RF-4  /  Sit.:  War  -  Flights  Executor  Event  Graph 
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Figure  23  shows  the  event  graph  of  the  ReportsGenerator  component. 
Event  in  red  is  a  changed  event  specific  to  the  war  scenario.  The  rest  of  the  events  are  the 
same  as  in  the  peace  scenario.  The  FCEffect  event  triggers  the 
UpdateFlightsForFCDefect  event  of  FlightsPlanner  component. 


Parameters 

totalNumberAnalysts  =  Number  of  analysts 
ffcReparrime  =  Camera  repair  time 
W>rtw>*ingr»»  =  Report  writing  time 


State  Variables 

numberAvailableAnalysts=  Number  of  available  analysts 
numberProvidedRecRequests  =  Number  of  provided  requests 
recRequestsQueue  =  Queue  for  rec.  request 


Figure  23.  Conf :  RF-4  /  Sit.:  War  -  Reconnaissance  Request  Report  Generator  Event 

Graph 


2.  Configuration  of  the  F-16 


a.  Situation:  Peace 

Figure  24  shows  the  main  components  of  the  F-16  configuration  in  the 
peace  situation  model.  There  are  five  main  components  in  the  model: 
RecSquadronWorkflow,  RecRequestCreator,  FlightsPlanner,  FlightsExecutor  and 
ReportsGenerator.  These  main  components  are  independent  of  each  other,  but  are 
connected  by  using  listener  and  adapter  patterns  to  make  the  whole  model  run  accurately. 

Each  main  component  is  explained  below  in  detail. 
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Figure  24.  Conf.:  F-16  /  Sit.:  Peace  -  Components  with  Listener  and  Adapter  Patterns 


The  RecSquadronWorkflow  component  of  the  F-16  configuration  peace 
situation  scenario  is  same  as  in  the  RF-4  configuration  peace  situation  scenario. 

Figure  25  shows  the  event  graph  of  the  RecRequestsCreator  component. 
The  RecRequestsCreator  component  of  the  F-16  configuration  peace  situation  scenario  is 
almost  the  same  as  in  the  RF-4  configuration  peace  situation  scenario. 
RecRequestsCreator  component  of  F-16  configuration  peace  situation  scenario  do  not 
have  RemovefromArchive  events,  and  they  use  the  FilmingType  class  (which  has 
updatedRepRequired  and  addToArchive  extra  fields)  instead  of  CameraType  class  as  one 
of  the  fields  for  the  RecRequest  class. 
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Figure  25.  Conf.:  F-16  /  Sit.:  Peace  -  Reconnaissance  Requests  Creator  Event  Graph 

Figure  26  shows  the  event  graph  of  the  FlightsPlanner  component.  The 
FlightsPlanner  component  plans  reconnaissance  flights  dynamically  based  on  availability 
of  resources  such  as  aircrafts,  pilots,  and  pods  (cameras).  During  flight  planning  it 
combines  reconnaissance  requests  that  have  adjacent  targets  into  one  flight  and  calculates 
flight  durations.  FlightsPlanner  also  records  successful  reconnaissance  request  mission 
target  locations  to  the  relevant  target  location  archive,  and  removes  obsolete  target 
locations  from  the  relevant  target  location  archive.  FlightsPlanner  keeps  track  of: 

•  Aircraft,  pilot,  and  camera  inventories. 

•  Planned  flights  list. 

•  Two  separate  queues  for  day  and  night  reconnaissance  requests. 

•  Two  separate  archives  (optic  and  infrared)  for  storing  target  locations  filmed  by 
reconnaissance  missions. 

In  the  RecRequestArrival  event  the  arrival  time  of  reconnaissance  request 
(r)  is  stamped,  and  then  the  numberRecRequestArrivals  state  variable  is  incremented  by 
one.  If  the  reconnaissance  request  does  not  require  an  up-to-date  image  of  the  target  and 
the  relevant  target  location  archive  (al)  contains  the  target  of  reconnaissance  request  (Pt), 
the  RecRequestArrival  event  schedules  the  ReEvalRequired  event  with  a 
ttimeBetArrivaiAndworkingstart  delay.  Otherwise  the  reconnaissance  request  (r)  is  added  to  the 
relevant  queue  and  the  RecRequestArrival  event  schedules  the  RemoveRecRequest  event 
with  tdueDate  -  simTime  delay.  Each  reconnaissance  request  should  be  responded  to  before  its 
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due  date  passes.  The  RemoveRecRequest  event  removes  reconnaissance  requests  from 
the  relevant  queue  and  increments  the  number RemovedRecRequests  state  variable  by  one. 

If  the  target  of  the  reconnaissance  request  (Pt)  is  going  to  be  added  to  the 
relevant  target  location  archive  (al),  the  RecReportProvided  event  schedules  the 
AddToArchive  event  and  cancels  the  RemoveRecRequest  event  for  the  relevant 
reconnaissance  request  (r).  The  RecReportProvided  event  also  updates  the 
totalTimeAtWorkFlow  state  variable. 

If  the  relevant  target  location  archive  (al)  contains  the  target  location  (Pt), 
the  AddToArchive  event  cancels  the  RemoveFromArchive  event  for  that  target  location. 
Otherwise,  the  target  location  (Pt)  is  added  to  the  relevant  target  location  archive  (al). 
The  AddToArchive  event  also  schedules  the  RemoveFromArchive  event  with 
treportUptoDateTime  delay  for  Same  target  location  (Pt).  The  RemoveFromArchive  event 
removes  the  target  location  (Pt)  from  relevant  target  location  archive  (al). 

The  PlanFlights  event  makes  the  main  flight  planning  based  on  availability 
of  cameras,  aircrafts  and  pilots  within  daily  flight  hours  for  both  day  and  night  flights. 
During  flight  planning  it  combines  reconnaissance  requests  that  have  adjacent  targets  into 
one  flight  (f)  and  calculates  flight  durations.  The  PlanFlights  event  schedules  the 
StartFlight  events  for  planned  flights  with  relevant  tpiannedFiightiime  -  simiime  delays. 

The  duration  of  a  flight  is  calculated  the  same  as  in  the  RF-4  configuration 
peace  situation  model. 

The  UpdateFlightsForFPDefect  event  first  cancels  the  flights  (f)  which 
cannot  be  completed  at  their  scheduled  times  because  of  camera  defect(s)  and  tries  to 
plan  new  flights  for  those  canceled  ones  by  using  the  planNewFlight  method. 

The  UpdateFlightsForACDefect  event  first  cancels  the  flights  (f)  which 
cannot  be  completed  at  their  scheduled  times  because  of  aircraft  defect(s)  and  tries  to 
plan  new  flights  for  those  canceled  ones  by  using  the  planNewFlight  method. 
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The  PlanNewFlight  event  tries  to  plan  a  new  flight  for  its  argument  flight 
(f)  by  using  the  planNewFlight  method.  It  also  adds  the  reconnaissance  requests  of  the 
cancelled  or  unsuccessful  flight  to  the  relevant  queue. 

The  planNewFlight  private  method  tries  to  plan  a  new  flight  (nF)  for  a 
cancelled  or  unsuccessful  flight  by  using  available  resources  (cameras,  aircraft,  and 
pilot).  If  a  new  flight  can  be  planned  it  schedules  the  StartFlight  event  with  a 
tpiannedFiightTime-simTime  delay  and  Updates  the  flights  list. 

The  JoinFlightPodsPool  event  updates  cameras’  availability  times.  The 
JoinPilotPool  event  updates  pilots’  availability  times.  The  JoinAirCraflsPool  event 
updates  aircraft  availability  times. 

The  Updateinventories  event  updates  inventories  of  cameras,  pilots,  and 
aircraft  after  an  aircraft  takes  off  for  a  reconnaissance  mission. 

The  RemoveFlightAndRecRequests  event  updates  the  flights  list  and 
relevant  reconnaissance  request  queue  after  an  aircraft  takes  off  for  a  reconnaissance 
mission. 
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WwtupToodte  ”  Report  is  valid  within  this  time 
WnoatAirAndws  =  Duratiop  up  to  TGevaluation  task 
tduoDJtoafcnTiTio  =  Time  up  to  due  date  of  rec.  request 
=  Time  up  to  Planned  flight 
time  of  rec.  request 


flightList  =  List  of  planned  flights 

daylightRecRequestsQueue  =  Queue  for  day  flight  requiring  requests 
nightRecRequestsQueue  =  Queue  for  night  flight  requiring  requests 
numberRecRequestArrivals  =  Number  of  arrived  rec.  requests 
numberRemovedRecRequests  =  Number  of  non-responded  rec.  requests 
totalTimeAtWorkFlow  =  Duration  between  rec.  request  arrival  and  its 

report  generation 


Figure  26.  Conf.:  F-16  /  Sit.:  Peace  -  Reconnaissance  Flights  Planner  Event  Graph 


Figure  27  shows  the  event  graph  of  the  FlightsExecutor  component.  The 
FlightsExecutor  component  deals  with  reconnaissance  flights,  after  flight  operations,  and 
most  of  the  state  variable  changes  in  the  whole  model. 

First,  the  TakeOff  event  schedules  the  UpdateQueueAndList  event  with 
the  relevant  flight  (f)  and  resource  group  (camera,  aircraft,  and  pilot)  parameters. 

Second,  the  TakeOff  event  schedules  the  AssignResources  event  with 
resource  group  (rG)  parameter.  Unless  the  simulation  time  is  less  than  the  availability 
times  of  any  resource  group  component  (camera,  aircraft  and  pilot),  the  TakeOff  event 
decrements  relevant  numberAvailableFlightCameras,  numberAvailableAirCrafts  and 
numberAvailablePilots  state  variables  by  one. 
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Third,  the  TakeOff  event  generates  a  random  number  for  aircraft  defect 
probability  (p).  If  p  is  less  than  or  equal  to  user  input  for  aircraft  defect  probability,  the 
TakeOff  event  schedules  LandForACDefect  by  passing  the  relevant  flight  (f)  with  a 
taircraftoefectRecogTime  delay.  If  p  is  greater  than  the  user  input  for  aircraft  defect  probability, 
then  the  TakeOff  event  schedules  the  Land  event  by  the  passing  relevant  flight  (f)  with  a 
tfiightDuration  delay  and  generates  a  random  number  for  data  link  defect  probability  (q).  If  q 
is  less  than  or  equal  to  user  input  for  data  link  defect  probability,  the  TakeOff  event 
schedules  the  StartMissionEvaluation  event  by  passing  the  relevant  flight  (!)  and  true 
with  a  tfiightDuration  +  avgEngstopTime  delay.  If  q  is  greater  than  user  input  for  data  link  defect 
probability,  the  TakeOff  event  schedules  the  StartDLCom  event  by  passing  the  relevant 

flight  (f)  with  a  tfiightDuration  -  landDLCom  delay . 

Finally,  if  the  simulation  time  is  less  than  the  availability  times  of  any 
resource  group  component  (camera,  aircraft,  and  pilot)  the  TakeOff  event  schedules  the 
FlightCancelled  event  by  passing  the  relevant  flight  (f)  as  a  parameter. 

The  Land  event  schedules  the  StartACMaintenance  event  by  passing  the 
relevant  aircraft  (a)  with  a  tavgEngstopTime  delay.  Then  it  schedules  the  NewAvailablePilot 
event  by  passing  the  relevant  pilot  (p)  with  trestTimeForPiiots  delay. 

The  StartDLCom  event  schedules  the  StartMissionEvaluation  event  by 
passing  relevant  flight  (f)  and  false  with  tDEComiime  delay. 

The  LandForACDefect  event  schedules  four  events.  First,  it  schedules  the 
NewAvailableFlighPod  event  by  passing  the  relevant  pod  (fp)  with  a  tavgEngstopiime  delay. 
Second,  it  schedules  the  NewAvailablePilot  event  by  passing  the  relevant  pilot  (p).  Third, 
it  schedules  the  StartACDefectRepair  event  by  passing  relevant  aircraft  (a)  with  a 
tavgEngstopTime  delay.  Fourth,  it  schedules  the  FlightCancelled  event  by  passing  relevant 
flight  (f)  with  a  tavgEngstopTime  delay. 

The  StartACDefectRepair  event  updates  aircraft  inventory.  Then  it 
schedules  the  ACDefectEffect  event.  Finally,  it  schedules  the  EndACDefectRepair  event 
by  passing  relevant  aircraft  (a)  with  a  tairCraftoefRepTime  delay. 
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The  EndACDefectRepair  event  schedules  the  NewAvailableAirCraft  event 
by  passing  relevant  aircraft  (a).  The  StartACMaintenance  event  updates  aircraft 
inventory.  Then  it  schedules  the  EndACMaintenance  event  by  passing  relevant  aircraft 
(a)  with  tavgMaintTimeForAirCrafts  delay.  The  EndACMaintcnance  event  schedules  the 
NewAvailableAirCraft  event  by  passing  relevant  aircraft  (a). 


The  NewAvailableFlightPod  event  increments  the  relevant 
numberAvailableFlightPods  state  variable  by  one.  The  NewAvailableAirCraft  event 
increments  the  numberAvailableAirCrafts  state  variable  by  one.  The  NewAvailablePilot 


event  increments  the  numberAvailablePilots  state  variable  by  one. 


totalNumberAirCrafts  =  Number  of  aircrafls  numberAvailableAirCrafts  =  Number  of  available  aircrafts 

totalNumberPilots  =  Number  of  pilots  numberAvailablePilots  =  Number  of  available  pilots 

Wio<nim  “  Fllflhl  time 

U<nnMwnn«nna  =  RecognlUon  of  aircraft  defect  time 
^EnnataaTM*  =  AverBoe  time  for  ertglne  shut  off 
Rwt  amo  for  pilots 
Aircraft  defect  repair  Bme 

VigiMiniMFaiMcnira=  Average  time  for  aircraft  maintenance 

twiioaiiiu'  tiMdOLOm  =  Aircraft  Is  in  line  of  sight  with  the  antenna  within  fols  dme 

Figure  27.  Conf :  F-16  /  Sit.:  Peace  -  Reconnaissance  Flights  Executor  Event  Graph 
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Figure  28  depicts  the  event  graph  of  the  ReportsGenerator  component. 
The  ReportsGenerator  component  deals  with  the  operations  after  successful 
reconnaissance  flights,  which  are  interpretation  of  the  images  and  writing  reconnaissance 
reports  based  on  their  requirements. 

The  InterpretFPImages  event  can  schedule  four  different  events.  First,  it 
generates  a  random  number  for  camera  defect  probability  (pi);  if  this  number  is  less  than 
or  equal  to  user  input  for  relevant  camera  defect  probability,  it  schedules  the 
StartFPDefectRepair  event  by  passing  relevant  pod  (fp).  Second,  if  pi  is  greater  than  the 
user  input  for  relevant  camera  defect  probability,  then  the  InterpretFPImages  event 
schedules  the  StartFPMaintenance  event  by  passing  relevant  pod  (fp). 

Third,  based  on  generated  random  numbers,  if  there  is  a  camera  defect  or 
bad  weather  condition,  then  the  InterpretFPImages  event  schedules  the  BadImageResults 
event  by  passing  relevant  flight  (I).  Fourth,  based  on  generated  random  numbers,  if  there 
is  no  camera  defect  and  no  bad  weather  condition,  then  the  InterpretFPImages  event 
generates  random  report  writing  times  for  reconnaissance  requests  in  the  relevant  flight 
(I)  and  adds  them  to  the  reconnaissance  requests  queue.  If  there  is  available  image 
analyst,  the  InterpretFPImages  event  schedules  the  StartReportWriting  event. 

The  StartReEval  event  adds  the  reconnaissance  request  (r)  to  the 
reconnaissance  requests  queue.  If  there  is  available  image  analyst,  it  schedules  the 
StartReportWriting  event.  The  StartFPMaintenance  event  updates  camera  inventory. 
Then  it  schedules  the  EndFPMaintenance  event  by  passing  the  relevant  pod  (Q))  with 

tavgMaintTimeForFPs  delay . 

The  EndFPMaintenance  event  schedules  the  NewAvailableFlightPod 
event  by  passing  relevant  pod  (^). 

The  StartFPDefectRepair  event  updates  camera  inventory.  Then  it 
schedules  the  FPDeffectEffect  event  by  passing  the  relevant  pod  (^).  Finally,  the 
StartFPDefectRepair  event  schedules  the  EndFPDefectRepair  event  by  passing  relevant 
pod  (^)  with  tflightPodRepairTime  delay.  The  EndFPDefectRepair  event  schedules  the 
NewAvailableFlightPod  event  by  passing  relevant  pod  (fp). 
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The  StartReportWriting  event  takes  the  first  reconnaissance  request  (r)  in 
the  queue,  removes  it  from  the  queue  and  schedules  the  EndReportWriting  event  by 
passing  that  reconnaissance  request  (r)  with  treportwritingiime  delay.  The  StartReportWriting 
event  also  decrements  the  number  Available  Analysts  state  variable  by  one. 

The  EndReportWriting  event  increments  the  numberProvidedRecRequests 
and  numberAvailableAnalysts  state  variables  by  one.  If  there  is  any  reconnaissance 
request  in  the  queue  waiting  for  the  report  writing  process,  the  EndReportWriting  event 
schedules  the  StartReportWriting  event. 


totalNumberAnalysts  =  Number  of  analysts  numberAvailableAnalysts=  Number  of  available  analysts 

^RBpvTiim  =  repair  time  numberProvidedRecRequests  =  Number  of  provided  requests 

WoriWrtingTirna  =  f^^popt  Wilting  time  recRequestsQueue  =  Queue  for  rec.  request 

^(MayTime  =  Duration  up  to  relevant  event 
UsAUn'nntRivFPa  =  Average  maintenance  time  for  pods 

Figure  28.  Conf :  F-16  /  Sit.:  Peace  -  Reconnaissance  Reports  Generator  Event  Graph 


b.  Situation:  War 


Figure  29  shows  the  main  components  of  the  RF-4  configuration  in  a  war 
situation  model.  There  are  five  main  components  in  the  model  like  peace  situation: 
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RecSquadronWorkflow,  RecRequestCreator,  FlightsPlanner,  FlightsExecutor  and 
ReportsGenerator.  Even  though,  during  the  peace  time,  training  is  done  according  to 
“Train  like  you  fight”  principle,  there  are  significant  changes  in  the  war  situation.  Events 
in  red  are  either  new  or  changed  events  specific  to  the  war  scenario.  The  rest  of  the  events 
are  the  same  as  in  the  peace  scenario.  Each  main  component  is  explained  below  in  detail. 


^ecRequestsCreator  FlightsPlanner 


Arrival 


RecSquadron 
Workflow 


A 


CreateRec 

Requests 


PlanFllghta 


L _ ]  Component 

I  Event 

I  Event  Listener 
^Adapter 


RecRequestArrival 


StartRIght 


RemoveFllghtAnd 

RecRequests 


UpdateRIghlsForPilot 

Effect 


JoinFlighPodsPod 


JoinAirCraflsPool 


JoInPllotsPool 


PlanNewFllght 


UpdateFllghtsFor 

ACEffect 


UpdateRIghtsFor 

FPEffect 


RecReportProvIded 


Updateinventories 


ReEvalReqiilred 


RemoveFrom  Inventories 


FlightsExecutor 


TakeOff 


UpdateQueueAndList 


PilotEffect 


NewAvailableFlighlPod 


NewAvaiJableAirCraft 


NewAvallablePllot 


NewRightRequired 


ACEffect 


StartMleelon 

Evaluatton 


FPEffect 


<  AssIgnResources 


ReaourceaLoet 


ReportGenerator 


NewAvailableFlightPod 


g  BadlmageReaulta 


IntarpretFPImages 


FPEffect 


EndReportWritIng 


^  StartReEv^ 


Figure  29.  Conf :  F-16  /  Sit.:  War  -  Components  with  Listener  and  Adapter  Patterns 

RecSquadronWorkflow  component  of  F-16  configuration  war  situation 
scenario  is  same  as  RF-4  configuration  peace  situation  scenario.  Only  the  value  for 
tpianfiightsTime  time  delay  is  incremented  two  hours  for  more  flight  hours. 

RecRequestsCreator  component  of  F-16  configuration  war  situation 
scenario  is  same  as  F-16  configuration  peace  situation  scenario.  Only  the  due  dates  given 
to  reconnaissance  requests  are  different.  In  peace  scenario  duration  between  arrival  and 
due  date  of  each  reconnaissance  request  is  random  but  in  war  scenario  this  duration  is  just 
one  day. 
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Figure  30  shows  the  event  graph  of  the  FlightsPlanner  component.  Events 
in  red  are  either  new  or  changed  events  specific  to  the  F-16  configuration  war  scenario. 

The  UpdateFlightsForFPEffect  event  first  cancels  the  flights  which  cannot 
be  completed  at  their  scheduled  times  because  of  either  camera  defect(s)  or  camera  loss 
(due  to  aircraft  interception)  and  then  tries  to  plan  new  flights  for  those  canceled  ones  by 
using  the  planNewFlight  method. 

The  UpdateFlightsForACEffect  event  first  cancels  the  flights  which 
cannot  be  completed  at  their  scheduled  times  because  of  either  aircraft  defect(s)  or 
aircraft  loss  (due  to  aircraft  interception)  and  then  tries  to  plan  new  flights  for  those 
canceled  ones  by  using  the  planNewFlight  method. 

The  UpdateFlightsForPilotEffect  event  first  cancels  the  flights  which 
cannot  be  completed  at  their  scheduled  times  because  of  pilot  loss  (due  to  aircraft 
interception),  and  then  tries  to  plan  new  flights  for  those  canceled  ones  by  using  the 
planNewFlight  method. 

The  RemoveFromInventories  event  schedules  the  StopSimulation  event 
when  either  there  is  no  aircrafts,  cameras  or  pilots  left,  which  means  the  reconnaissance 
squadron  cannot  operate. 
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Figure  30.  Conf.:  F-16  /  Sit.:  War  -  Reconnaissance  Flights  Planner  Event  Graph 


Figure  31  shows  the  event  graph  of  the  FlightsExecutor  component. 
Events  in  red  are  either  new  or  changed  events  specific  to  the  F-16  configuration  war 
scenario.  The  rest  of  the  events  are  the  same  as  in  the  F-16  configuration  peace  scenario 
except  the  probability  of  reconnaissance  aircrafts’  interception  by  enemy  which  affects 
the  entire  reconnaissance  cycle  due  to  lost  resources  such  as  pilots,  aircrafts  and  cameras. 

The  TakeOff  event  generates  a  random  number  for  aircraft  defect 
probability  (p).  If  p  is  greater  than  the  user  input  for  aircraft  defect  probability,  then  the 
TakeOff  event  schedules  ArriveToTarget  event  by  passing  relevant  flight  (f)  with 
tflightDuration/2  delay.  If  the  simulation  time  is  less  than  availability  times  of  any  resource 
group  component  (camera,  aircraft  and  pilots)  the  TakeOff  event  schedules  the 
NewFlightRequired  event  by  passing  relevant  flight  (f). 

A  random  number  for  aircraft  intercept  probability  (p)  is  generated  and 
then  compared  to  the  user  input  for  aircraft  intercept  probability.  If  the  aircraft  intercept 
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probability  (p)  is  less  than  or  equal  to  the  user  threshold,  then  the  ArriveToTarget  event 
schedules  the  ACIntercepted  event.  If  not,  it  schedules  three  events.  First,  the 
ArriveToTarget  event  schedules  the  Land  event  with  tfiightDuration/2  delay.  Second,  it 
schedules  the  StartDLCom  event  if  the  data  link  error  (p)  is  greater  than  the  user 
threshold  PdataiinkError,  in  tfiightDuration/2  -  tLandDLCom  time  delay.  Finally,  it  schedules 
StartMissionEvaluation  if  the  data  link  error  (p)  is  less  than  or  equal  to  the  user  threshold, 

with  tavgEngStopTime  tflightDuration/2  time  delay. 

The  ACIntercepted  event  schedules  five  events.  First,  it  schedules  the 
ResourceLost  event  by  sending  pertinent  resource  group  (rG).  Second,  it  schedules  the 
NewFlightRequired  event  by  sending  relevant  flight  (f).  Third,  it  schedules  the  ACEffect 
event  by  sending  pertinent  aircraft  (a)  and  reason  code  (rC).  The  reason  code 
differentiates  aircraft  interception  from  aircraft  defects.  Fourth,  it  schedules  the  FPEffect 
event  by  sending  pertinent  flight  pod  (fp)  and  reason  code  (rC).  Finally,  the  PilotsEffect 
event  is  scheduled  by  sending  related  pilot  (p). 

The  LandForACDefect  event  schedules  the  NewFlightRequired  event  by 
passing  relevant  flight  (f)  with  taverageEngineStopTime  delay. 
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totalNumberAirCrafls  =  Number  of  aircrafts  numberAvailableAirCrafts  =  Number  of  available  aircrafts 

totalNumberPilots  =  Number  of  pilots  numberAvailablePilots  =  Number  of  available  pilots 

~  Flight  time  until  target/land 
Wnw>nfntfTnrriaT¥nn  =  Recognition  of  aircraft  defect  time 
WvgEngsk^Time  =  Average  time  for  engine  shut  off 
Win0fafPfcte=  Rost  time  for  pilots 
^^cnMMRapTkna-  Aircraft  defect  repair  time 
^gutfiianTiniroiAkCniita"  Average  time  for  aircraft  maintenance 

-  k^DLCom  ~  Aircraft  is  in  iine  of  sight  with  the  antenna  within  this  time 

touoonTini  =  Averago  data  comunicaton  time 

UgEr«skipTiff«  WiDumHof/2  =  Mlssioii  evalustion  start  time  if  Datalink  doesn't  work 


Figure  31.  Conf.:  F-16  /  Sit.:  War  -  Reconnaissance  Flights  Executor  Event  Graph 


Figure  32  shows  the  event  graph  of  the  ReportsGenerator  component. 
Event  in  red  is  a  changed  event  specific  to  the  war  scenario.  The  rest  of  the  events  are  the 
same  as  in  the  peace  scenario.  The  FPEffect  event  triggers  the  UpdateFlightsForFPDefect 
event  of  FlightsPlanner  component. 
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totalNumberAnatysts  =  Number  of  analysts 
tiPR^idriim.  =  Pod  repair  time 
WiNribngTim.  =  Roport  WTltifig  time 
^(MayTime  =  Duration  up  to  relevant  event 
^ManrmBForFPa  =  Average  maintenance  time  for  pods 


numberAvailableAnalysts=  Number  of  available  analysts 
numberProvidedRecRequests  =  Number  of  provided  requests 
recRequestsQueue  =  Queue  for  rec.  request 


Figure  32.  Conf.:  F-16  /  Sit.:  War  -  Reconnaissance  Report  Generator  Event  Graph 
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IV.  DESIGN  OF  EXPERIMENTS 


Simulation  analysts  and  their  clients  might  seek  to  (i)  develop  a  basic 
understanding  of  a  particular  simulation  model  or  system,  (ii)  find  robust 
decisions  or  policies,  or  (iii)  compare  the  merits  of  various  decisions  or 
policies  (Kleijnen  et  ah,  2005). 

Simulation  experiments  are  needed  to  provide  solutions  for  these  aims  listed 
above  that  are  all  related  to  our  research  questions. 

There  are  many  classic  experimental  designs.  When  the  number  of  factors  is 
large,  more  effective  and  efficient  designs  are  required.  Sanchez  (2008)  says,  “A  well- 
designed  experiment  allows  the  analyst  to  examine  many  more  factors  than  would 
otherwise  be  possible,  while  providing  insights  that  cannot  be  gleaned  from  trial-and- 
error  approaches  or  by  sampling  factors  one  at  a  time.”  One  more  benefit  of  well- 
designed  experiments  is  getting  insight  and  information  in  a  relatively  short  amount  of 
time  (Sanchez,  2008). 

A.  INPUT  FACTORS 

One  of  the  initial  steps  an  experimenter  must  take  to  design  a  good  experiment  is 
identify  the  experimental  factors.  Factors  are  the  input  (or  independent)  variables  that 
might  have  some  impact  on  responses  (i.e.,  experimental  outputs).  Generally,  an 
experiment  might  have  many  factors,  each  of  which  might  get  a  variety  of  values,  called 
levels  of  the  factor  in  DOE  (Design  of  Experiments)  terminology.  Identifying  which  of 
the  factors  are  really  important  for  which  responses,  and  which  are  not  and  can  thus  be 
dropped  from  further  consideration,  greatly  reduces  the  experimental  effort  and  simplifies 
the  task  of  interpreting  the  results  of  experiment  (Sanchez,  2008). 

All  of  the  input  factors  (decision  and  noise  factors)  of  interest  for  different 
scenarios  of  Reconnaissance  Squadron  Workflow  are  shown  in  Table  2  and  described 
below.  These  factors  are  anticipated,  a  priori,  to  be  the  factors  with  the  greatest  influence 
the  Reconnaissance  Squadron  Workflow  and  so  on  the  measures  of  effectiveness.  Input 
and  distributional  parameters  are  based  on  authors’  experience,  and  are  used  to  generate 
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the  minimum  and  maximum  levels  of  eaeh  input  faetor  for  this  study.  Values  of  the  input 
faetors  used  in  each  experiment  are  presented  in  the  Appendix. 


Ntim 

Parameters 

Peace 

War 

Peace 

War 

RF-4 

RF-4 

F-16 

F-16 

1 

Decision 

numTlPods 

« 

• 

2 

numT2Pods 

* 

3 

numTlCam 

* 

4 

nunnT2Cam 

* 

5 

numT3Cam 

* 

6 

numT4Cam 

* 

« 

7 

nunnT5Cam 

* 

8 

numACs 

* 

« 

* 

9 

numPilots 

* 

« 

« 

* 

10 

numAnaIvsts 

* 

« 

* 

* 

11 

Noise 

modeNumReq 

« 

« 

* 

« 

12 

halfDistNumRea 

« 

« 

* 

« 

13 

modeHiqhPriDueDays 

* 

14 

halfDistHighPriDueDays 

* 

15 

priDueDaysLowOyerHiqhRatio 

« 

16 

ooImqTProb 

* 

17 

tlCamsDefProb 

* 

18 

t2CamsDefProb 

* 

19 

t3CamsDefProb 

* 

20 

t4CamsDefProb 

* 

« 

21 

t5CamsDefProb 

* 

« 

22 

tlPodsDefProb 

« 

23 

t2PodsDefProb 

« 

24 

aCDefProb 

* 

« 

« 

« 

25 

OilotFilErProb 

* 

« 

26 

badWeatConProb 

* 

• 

27 

upRepReqProb 

* 

• 

28 

dataLinkDefProb 

* 

* 

29 

yerAnqTProb 

* 

* 

30 

niqhtFliqhtProb 

* 

* 

31 

aCInterceptProb 

* 

* 

32 

modeACRepTime 

* 

* 

* 

* 

33 

halfDistACRepTime 

* 

* 

* 

34 

m  ode  Pod  s  RepTi  me 

* 

* 

35 

halfDistPodsRepTime 

* 

* 

36 

modeTacCamsRepTime 

* 

« 

37 

halfDistTacCamsRepTime 

* 

« 

38 

camsRepTimeStraOverTacRatio 

* 

Total 

17 

25 

21 

11 

Table  2.  Input  Factors  used  in  the  Models 


1.  Number  of  Type  1  Pods 

Type  1  pods  are  used  for  day  and  night  flights  on  the  F- 16  aircrafts.  Their  image 
type  is  both  optic  and  IR,  and  the  angle  type  is  vertical. 

2.  Number  of  Type  2  Pods 

Type  2  pods  are  used  for  day  and  night  flights  on  the  F-16  aircrafts.  Their  image 
type  is  both  optic  and  IR,  and  the  angle  type  is  oblique. 
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3. 


Number  of  Type  1  Cameras 


Type  1  cameras  are  used  for  daytime  flights  on  the  Rf’-4  aircrafts.  Their  image 
type  is  optic  and  the  angle  type  is  vertical. 

4.  Number  of  Type  2  Cameras 

Type  2  cameras  are  used  for  night  flights  on  the  RF-4  aircrafts.  Their  image  type 
is  optic  and  the  angle  type  is  vertical. 

5.  Number  of  Type  3  Cameras 

Type  3  cameras  are  used  for  daytime  flights  on  the  RF-4  aircrafts.  Their  image 
type  is  optic  and  the  angle  type  is  vertical. 

6.  Number  of  Type  4  Cameras 

Type  4  cameras  are  used  for  night  flights  on  the  RF-4  aircrafts.  Their  image  type 
is  infrared  and  the  angle  type  is  vertical. 

7.  Number  of  Type  5  Cameras 

Type  5  cameras  are  used  for  daytime  flights  on  the  RF-4  aircrafts.  Their  image 
type  is  optic  and  the  angle  type  is  oblique. 

8.  Number  of  Aircraft 

There  are  two  types  of  aircraft  used  for  different  scenarios:  the  RF-4  and  F-16. 

9.  Number  of  Pilots 

For  flight,  the  RF-4  aircraft  requires  two  pilots  and  the  F-16  aircraft  requires  one 
pilot  to  do  reconnaissance  missions. 
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10.  Number  of  Image  Analysts 


Image  analysts  interpret  and  evaluate  films  when  the  films  arrive.  They  do  this  job 
until  all  of  the  films  in  the  queue  are  evaluated. 

1 1 .  Number  of  Reconnaissance  Requests 

This  input  factor  corresponds  to  number  of  reconnaissance  requests  arriving  to  the 
squadron  every  day.  A  symmetric  triangular  distribution  is  used  for  this  input  factor. 

12.  Half  Distance  Number  of  Reconnaissance  Requests 

This  input  factor  is  used  for  calculating  the  number  of  reconnaissance  requests 
triangular  distribution’s  minimum  and  maximum  values  by  subtracting  from  and  adding 
to  the  number  of  reconnaissance  requests,  respectively. 

13.  Number  of  Due  Days  for  Reconnaissance  Requests 

They  are  five  types  of  priorities  for  reconnaissance  requests:  "LOWEST," 
"LOW,"  "DEFAULT,"  "HIGH,"  and  "HIGHEST."  Priorities  of  reconnaissance  requests 
are  generated  randomly,  by  using  a  discrete  integer  random  variable.  In  our  model, 
"HIGH"  or  "HIGHEST"  priority  reconnaissance  requests  are  high  priority  and  the  other 
ones  are  low  priority  reconnaissance  requests.  Two  symmetric  triangular  distributions  are 
used  to  assign  the  mode  of  the  number  of  due  days  for  high  and  low  priority 
reconnaissance  requests  in  peace  situation  scenarios.  In  war  situation  scenarios,  the 
number  of  due  days  for  all  reconnaissance  requests  is  always  one. 

14.  Half  Distance  Number  of  Due  Days  for  Reconnaissance  Requests 

This  input  factor  is  used  for  calculating  the  number  of  due  days  for 
reconnaissance  requests  triangular  distribution’s  minimum  and  maximum  values  by 
subtracting  from  and  adding  to  the  mode  of  the  number  of  due  days  for  reconnaissance 
requests. 


58 


15.  Ratio  of  Low  Priority  Due  Days  Over  High  Priority  Due  Days 


This  input  factor  is  used  to  calculate  due  days  for  low  priority  reconnaissance 
requests  in  both  the  RF-4  and  F-16  configurations  and  only  in  peace  situation  scenarios. 
It  is  used  as  a  scalar  multiplier.  For  example,  the  mode  of  the  number  of  due  days  for 
low  priority  reconnaissance  requests  is  equal  to  this  scalar  factor  multiplied  by  the  mode 
of  the  number  of  due  days  for  high  priority  requests.  A  similar  process  is  used  to 
compute  the  half  distance  number  of  due  days  for  low  priority  reconnaissance  requests. 

16.  Optic  Image  Type  Probability 

A  reconnaissance  request’s  image  type  can  be  either  optic  or  infrared.  A  discrete 
integer  random  variable  whose  parameters  are  set  by  this  input  factor  is  used  to  assign 
reconnaissance  requests’  image  types. 

17.  Type  1,  2, 3, 4  and  5  Cameras’  Defect  Probabilities 

Different  types  of  cameras  have  different,  or  sometimes  nearly  the  same,  defect 
probabilities.  Discrete  Bernoulli  random  variables  whose  parameters  are  set  by  using 
these  input  factors  are  used  to  decide  whether  or  not  there  is/are  camera  defect(s)  after 
reconnaissance  missions. 

18.  Type  1,  2  Pods’  Defect  Probabilities 

Different  types  of  pods  have  different,  or  sometimes  nearly  the  same,  defect 
probabilities.  Discrete  Bernoulli  random  variables  whose  parameters  are  set  by  using 
these  input  factors  are  used  to  decide  whether  or  not  there  is/are  pod  defect(s)  after 
reconnaissance  missions. 

19.  Aircraft  Defect  Probabilities 

The  RF-4  aircraft  has  a  higher  defect  probability  than  the  F-16  aircraft.  Discrete 
Bernoulli  random  variables  whose  parameters  are  set  by  using  these  input  factors  are 
used  to  decide  whether  there  is/are  aircraft  defect(s)  in  a  small  amount  of  time  after 
takeoff  for  a  reconnaissance  mission. 
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20.  Pilot  Filming  Error  Probability 


A  discrete  Bernoulli  random  variable  whose  parameter  is  set  by  using  this  input 
factor  is  used  to  decide  whether  there  is  pilot  filming  error  during  target  filming. 

21.  Bad  Weather  Condition  Probability 

A  discrete  Bernoulli  random  variable  whose  parameter  is  set  by  using  this  input 
factor  is  used  to  decide  whether  there  is  a  bad  weather  condition  during  target  filming. 

22.  Requiring  Update  Reconnaissance  Mission  Probabilities 

A  discrete  Bernoulli  random  variable  whose  parameter  is  set  by  using  this  input 
factor  is  used  to  decide  whether  reconnaissance  requests  can  be  responded  to  from 
previous  missions  or  not.  This  parameter  is  used  only  in  the  F-16  configuration  models. 

23.  Data  Link  Defect  Probability 

In  the  F-16  configuration  models,  target  imagery  can  be  downloaded  using  a  data 
link.  The  corresponding  input  factor  is  a  discrete  Bernoulli  random  variable  that  is  used 
to  decide  whether  or  not  a  data  link  is  working. 

24.  Requiring  Vertical  Angle  Reconnaissance  Mission  Probability 

Reconnaissance  requests  can  require  vertical  or  oblique  imagery.  This  parameter 
is  used  for  deciding  which  requests  are  vertical  or  oblique. 

25.  Requiring  Night  Flight  Reconnaissance  Mission  Probabilities 

In  the  F-16  configuration  models,  IR  imagery  requiring  requests  can  be  flown 
both  in  day  and  night  hours.  This  parameter  is  used  for  deciding  which  IR  imagery 
requiring  reconnaissance  missions  are  flown  during  night  flight  time  or  during  day  flight 
time. 
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26.  Aircrafts’  Interception  Probabilities 


The  RF-4  aircraft  has  a  higher  interception  probability  than  the  F-16  aircraft. 
Discrete  Bernoulli  random  variables  whose  parameters  are  set  by  using  these  input 
factors  are  used  to  decide  whether  the  aircraft  was  intercepted  during  entrance  to  the 
target  area  for  a  reconnaissance  mission. 

27.  Aircraft  Repair  Time 

The  RF-4  aircraft  has  a  different  defect  repair  time  than  the  F-16  aircraft.  These 
defect  repair  times  are  short  in  war  situation  scenarios.  A  symmetric  triangular 
distribution  is  used  to  assign  aircraft  defect  repair  time  after  recognized  aircraft  defect(s). 

28.  Half  Distance  Aircraft  Repair  Time 

This  input  factor  is  used  for  calculating  the  aircraft  repair  time  triangular 
distribution’s  minimum  and  maximum  values  by  subtracting  from  and  adding  to  the 
aircraft  repair  time. 

29.  Pods’  Repair  Times 

Type  1  and  Type  2  pods  used  on  F-16  aircrafts  have  similar  defect  repair  times. 
Two  symmetric  triangular  distributions  are  used  to  generate  pod  defect  repair  times  after 
recognized  pod  defect(s). 

30.  Half  Distance  Pod’s  Repair  Time 

This  input  factor  is  used  for  calculating  the  pod  repair  time  triangular 
distribution’s  minimum  and  maximum  values  by  subtracting  from  and  adding  to  the  pod 
repair  time. 

31.  Tactic  Camera  Repair  Time 

Type  1,  2,  3  and  4  cameras  (tactic)  used  on  RF-4  aircrafts  have  a  different  defect 
repair  time  than  Type  5  (strategic)  camera  used  on  RF-4  aircrafts.  Two  symmetric 
triangular  distributions  are  used  to  generate  camera  defect  repair  times  after  recognized 
camera  defect(s). 
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32.  Half  Distance  Tactic  Camera  Repair  Time 


This  input  factor  is  used  for  calculating  tactic  camera  repair  time  triangular 
distribution’s  minimum  and  maximum  values  by  subtracting  from  and  adding  to  the  tactic 
camera  repair  time. 

33.  Ratio  of  Strategic  Camera  Repair  Time  Over  Tactic  Camera  Repair 
Time 

This  input  factor  is  used  to  calculate  the  repair  time  of  strategic  cameras  in  RF-4 
configurations.  It  is  used  as  a  scalar  with  tactic  camera  repair  time  and  half  distance  tactic 
camera  repair  time  input  factors. 

B.  PERFORMANCE  MEASURES 

As  previously  discussed,  there  are  two  performance  measures: 

1.  Average  Time  at  Workflow  for  Responded  Reconnaissance  Requests 

This  MoE  shows  the  mean  time  elapsed  after  the  creation  of  a  reconnaissance 
request  to  the  provided  report  for  that  reconnaissance  request.  This  value  should  be  small 
for  an  effective  Reconnaissance  Squadron  Workflow  configuration. 

2.  Removed  Over  Arrived  Reconnaissance  Requests  Ratio 

This  MoE  shows  the  ratio  of  reconnaissance  requests  not  responded  to  (for 
reasons  such  as  due  date  and  available  resources)  over  total  arrived  reconnaissance 
requests.  This  value  also  should  be  small  for  an  effective  Reconnaissance  Squadron 
Workflow  configuration. 

C.  DESIGN 

A  design  is  a  matrix  where  every  column  corresponds  to  a  factor,  and  the  entries 
within  the  column  are  settings  for  this  factor.  Each  row  represents  a  particular 
combination  of  factor  levels  and  is  called  a  design  point.  If  the  row  entries  correspond  to 
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the  actual  settings  that  will  be  used,  these  are  called  natural  levels.  Table  3  shows  a 
simple  design  in  natural  levels  that  could  be  used  for  an  experiment  involving  two 
factors. 


Natural  Levels 

Design 

Point 

^2 

1 

16 

20 

2 

IS 

20 

3 

16 

22 

4 

IS 

22 

5 

16 

24 

6 

IS 

24 

Table  3.  Experimental  Design  in  Natural  Levels  [After  (Sanchez,  2008)] 

As  Sanchez  (2008)  says,  “selecting  a  design  is  an  art,  as  well  as  a  science.”  The 
number  of  factors  and  the  mix  of  different  factor  types  (binary,  qualitative  or  discrete 
with  a  limited  number  of  levels,  discrete  with  many  levels,  or  continuous)  play  important 
roles  for  experimental  designs.  A  desirable  property  for  an  experimental  design  is 
orthogonality,  which  means  the  pairwise  correlation  between  any  two  columns  (factors) 
is  equal  to  zero.  An  orthogonal  design  makes  the  analysis  of  the  output  (Y’s)  we  get  from 
running  our  experiment  simple,  because  estimates  of  the  factors’  effects  and  their 
contribution  to  the  explanatory  power  (R^)  of  the  regression  metamodel  will  not  depend 
on  what  other  explanatory  terms  are  present  in  the  regression  metamodel  (Sanchez, 
2008). 

Two  designs  (one  for  decision  and  one  for  noise  factors)  were  generated  for  each 
scenario  (model)  by  using  “Generating  and  Improving  Orthogonal  Designs  by  Using 
Mixed  Integer  Programming  tool”  coded  by  Helcio  Vieira  Junior  (see  also  Vieira  et  ah, 
2010).  These  two  designs  were  crossed  by  using  Paul  Sanchez’s  cross. rb  program  to 
generate  designs  for  each  scenario.  Each  scenario’s  crossed  design  has  5,000  design 
points  which  gave  them  much  of  the  space-filling  and  orthogonality  properties  of 
factorial  designs  with  fine  grids,  but  requiring  orders  of  magnitude  less  sampling.  Figure 
33  shows  a  correlation  table  and  a  scatter-plot  matrix  built  in  JMP  version  8.0  (SAS 
Institute  Inc.,  2008)  software  for  some  of  the  input  factors  of  the  RF-4  configuration 
peace  situation  scenario.  As  seen  from  the  figure,  crossed  design  is  notably  good  at 
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space-filling  property  and  represents  many  eombinations  of  input  faetors.  It  also  shows 
that  pairwise  eorrelations  of  input  factors  are  almost  zero. 


I  Multivariate  || 


numT  1  Cams  numT2Cams  numTSCams  numT4Cams  numTSCams  numACs  numPilots  numAnalysts 


numTICams 

1.0000 

-0.0002 

0.0000 

-0.0016 

-0.0000 

-0.0012 

-0.0030 

-0.0006 

numT2Cams 

-0.0002 

1.0000 

0.0000 

0.0038 

-0.0000 

0.0002 

-0.0013 

0.0013 

numT3Cams 

0.0000 

0.0000 

1.0000 

0.0000 

0.0000 

0.0000 

0.0031 

0.0000 

numT4Cams 

-0.0016 

0.0038 

0.0000 

1.0000 

-0.0000 

0.0016 

0.0040 

0.0030 

numT5Cams 

-0.0000 

-0.0000 

0.0000 

-0.0000 

1.0000 

0.0000 

0.0000 

-0.0000 

numACs 

-0.0012 

0.0002 

0.0000 

0.0016 

0.0000 

1.0000 

0.0030 

0.0006 

numPilots 

-0.0030 

-0.0013 

0.0031 

0.0040 

0.0000 

0.0030 

1.0000 

0.0001 

numAnalysts 

-0.0006 

0.0013 

0.0000 

0.0030 

-0.0000 

0.0006 

0.0001 

1.0000 

The  correlations  are  estimated  by  Pairwise  method. 
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Figure  33.  Correlations  and  Scatter-plot  Matrix 


D.  SCENARIO  REPLICATION 

Simulation  models  eome  in  many  flavors.  There  are  deterministic 
simulations  (e.g.,  numerical  solutions  of  differential  equations,  where  the 
same  set  of  inputs  always  produces  the  same  output)  and  stochastie 
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simulations  (where  the  same  set  of  simulation  inputs  may  produee 
different  output  unless  the  random  number  streams  are  carefully 
controlled).  (Sanchez,  2008) 

Simulations  modeling  processes  that  occur  over  time  can  be  characterized  as 
terminating  or  non- terminating,  depending  on  the  stopping  conditions.  In  terminating 
simulations,  the  simulation  stops  after  either  a  pre-specified  amount  of  simulation  time 
has  elapsed,  or  when  a  specific  event  or  condition  occurs  (Sanchez,  2008).  Different 
scenarios  (models)  of  Reconnaissance  Squadron  Workflow  are  all  stochastic  and 
terminating.  To  study  these  models,  replication  should  be  used.  Each  repetition  of  the 
whole  design  matrix  is  called  a  replication  and  the  replications  are  independent.  We  have 
5,000  design  points  and  25  replications.  Then  the  total  number  of  experimental  units  for 
each  scenario  is  5,000  *  25  =  125,000  which  takes  approximately  seven  hours  to  run  on  a 
laptop  equipped  with  Intel(R)  2.26GHz  Core  2  Duo  processor  and  2GB  RAM. 

E.  DATA  DELETION 

Time  series  models  were  fit  by  using  JMP  software  to  predict  reconnaissance 
request’s  respond  times  (time  elapsed  after  creation  of  a  reconnaissance  request  to  the 
provided  report  for  that  reconnaissance  request)  for  all  of  the  scenarios.  We  saw  that  the 
magnitude  of  autocorrelation  drops  below  0.40  after  lag  1  and  stays  below.  Therefore,  we 
did  not  perform  any  data  deletion. 
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V.  ANALYSIS  AND  RESULTS 


This  chapter  presents  the  analysis  of  the  output  from  the  simulation  models 
discussed  in  Chapter  IV.  The  reader  should  note  the  fact  that  these  results  do  not 
constitute  a  definitive  solution  to  the  research  questions  for  specific  squadron  operations; 
although  we  use  data  based  on  authors’  experience,  the  actual  data  are  classified.  Future 
analysts  should  conduct  their  own  experiments  for  each  simulation  with  updated  inputs  to 
be  more  confident  in  the  answers  to  the  research  questions.  After  the  run  of  each 
simulation  experiment  is  over,  the  resultant  data  (.csv  files)  is  saved  to  the  hard  drive. 
Duplicates  of  headers  in  these  files  are  removed  by  using  stripheaderdups.rb  program 
written  by  Paul  Sanchez.  The  resultant  data  was  imported  into  JMP  software  for  analysis. 

A.  REGRESSION  ANALYSIS 

Mathematically,  let  Xi,  .  .  .  ,  Xk  denote  the  k  factors  in  an  experiment,  and  let  Y 
denote  a  response  of  interest.  Generally,  we  are  interested  in  constructing  response 
surface  metamodels  that  approximate  the  relationships  between  the  factors  and  the 
responses  with  statistical  models  (typically  regression  models).  First,  suppose  that  the 
Xi’s  are  all  quantitative,  although  they  can  be  discrete  or  continuous.  A  first-order  (main- 
effects)  model  means  we  assume: 

it 

=  ^0  +  5^  ^  r 

J=] 

where  the  s’s  are  independent  random  errors  with  mean  zero  (Sanchez,  2008). 

To  explore  any  quadratic  effects,  we  include  terms  like  Xi  as  potential 
explanatory  variables  for  Y.  For  two-way  interactions,  we  include  terms  like  X1X2.  A 
second-order  model  includes  quadratic  effects  and  two-way  interactions  as  below 
(Sanchez,  2008): 
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it  it  _ 

Y  =  ft)  +  £ftX,  +  £ft.,(X,-X,)- 

E=l  i=l 

i-l  it  _  _ 

+  E  E  Pi.i{Xi-X,){Xj-Xj}+e. 

i=]  j=i+] 

Regression  models  were  formulated  to  identify  the  impact  of  the  factors 
(explanatory  variables)  on  the  responses  (e.g.,  increasing,  linear,  quadratic),  and  whether 
the  levels  of  some  factors  influence  the  effects  that  other  factors  have  (called  factor 
interactions).  Fitting  regression  models  is  an  interactive  process.  Each  scenario’s 
experiment  data  was  summarized  (all  the  decision  and  noise  factors  as  the  group,  mean  of 
each  MoE  as  statistics)  in  JMP  software  to  fit  regression  models  and  partition  trees. 
Several  different  regression  models  were  fit  for  each  MoE  by  using  stepwise  regression 
analysis  in  JMP.  For  war  situation  scenarios,  regression  models  for 
avgTimeAtWFForResRecRequests  were  not  built  because  the  due  date  of  all  arriving 
reconnaissance  requests  is  one  day  after  they  arrive.  The  significance  thresholds  for 
entering  and  leaving  the  model  used  in  these  analyses  are  both  0.01  and  direction  is 
mixed.  Some  effective  regression  models  and  partition  trees  were  provided  in  the  relative 
sections  below. 

B.  ROBUST  DESIGN 

Robust  design  is  a  system  optimization  and  improvement  process  which  states 
that  a  system  should  not  be  evaluated  on  the  basis  of  mean  performance  alone.  In  addition 
to  giving  an  acceptable  mean  performance,  a  “good”  system  must  be  relatively 
insensitive  to  uncontrollable  sources  of  variation  present  in  the  system’s  environment  and 
nature.  The  purpose  of  robust  design  is  to  lead  to  better  decisions  in  terms  of 
implementation,  level  and  consistency  of  performance,  cost,  and  insight  into  the  drivers 
of  system  performance  (Sanchez,  2000). 

Factors  are  classified  as  decision  factors,  noise  factors,  or  artificial  factors.  The 
decision  factors  are  controllable  in  the  real  world  setting  modeled  by  the  simulation. 
Noise  factors  are  not  easily  controllable  or  are  controllable  only  at  great  expense  in  the 
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real  world  setting  sueh  as  aircraft  and  camera  defect  probabilities.  Artificial  factors  are 
simulation-specific  variables  such  as  initial  state  of  the  system,  termination  conditions 
and  random  number  streams  (Sanchez,  2000). 

For  performance  evaluation,  the  analyst  specifies  some  performance  characteristic 
of  special  interest,  and  an  associated  target  value  t.  The  cost  of  the  performance 
characteristic’s  fluctuation  around  the  target  value  is  measured  in  order  to  optimize  or 
improve  the  system.  An  ideal  configuration  would  result  in  the  performance 
characteristic’s  mean  equal  to  t  and  its  variance  equal  to  zero.  A  quadratic  loss  function  is 
a  common  way  to  trade  off  performance  mean  and  variability.  Let  x  and  Y(x)  denote  a 
vector  of  decision  factor  settings  and  the  associated  performance  characteristic, 
respectively.  Then,  assuming  no  loss  is  incurred  when  Y(x)  is  equal  to  t,  the  quadratic 
loss  function  can  be  written  as:  /(Y(x))  =  c[Y(x)  -  t\  where  c  is  the  scaling  constant  to 
convert  losses  into  monetary  units.  Using  the  quadratic  loss  function,  the  expected  loss 
associated  with  configuration  x  is  E(loss)  =  c[varY(x)  (py(x)  -  0^]  (Sanchez,  2000). 

Robust  design  has  two  important  benefits.  First,  because  of  a  chosen  system 
configuration’s  robustness,  it  is  likely  to  work  well  across  a  variety  of  realizations  of 
noise  factor  values.  Second,  there  will  be  an  improved  communication  between  the 
analyst  and  client  via  expected  loss  (Sanchez,  2000). 

To  find  out  robust  configurations  for  the  reconnaissance  squadron  in  different 
scenarios,  each  scenario’s  experiment  data  was  summarized  (all  the  decision  factors  as 
the  group,  mean  and  standard  deviation  of  MoE  as  statistics)  in  JMP.  The  target  value 
selected  for  remOverArrRecRequestsRatio  in  each  scenario  is  0.0.  The  target  value 
selected  for  avgTimeAtWFForResRecRequests  in  each  scenario  is  the  minimum 
observed  avgTimeAtWFForResRecRequests  during  a  relevant  experiment. 
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C.  OUTPUT  ANALYSIS  FOR  CONFIGURATION:  RF-4  /  SITUATION: 
PEACE 


I.  Average  Time  at  Workflow  for  Responded  Reconnaissance  Requests 


a.  Basic  Statistics 


Figure  34  shows  the  basic  statistics  for  the  average  time  at  workflow  for 
responded  reconnaissance  requests  (avgTimeAtWFForResRecRequests).  With  the 
realistic  values  used  by  the  authors,  the  mean  of  avgTimeAtWFForResRecRequests  is 
2.64  days  with  a  standard  deviation  of  0.886.  The  distribution  of 
avgTimeAtWFForResRecRequests  is  almost  symmetric  but  not  unimodal. 


avgTimeAtWFForResRecRequests 


Quantiles 

Moments 

100.0% 

maximum 

5.0532 

Mean 

2.6401497 

99.5% 

4.9256 

Std  Dev 

0.3355033 

97.5% 

4.3105 

Std  Err  Mean 

0.0125229 

90.0% 

3.7077 

Upper  95%  Mean 

2.6647001 

75.0% 

quartile 

3.3229 

Lower  95%  Mean 

2.6155993 

50.0% 

median 

2.5357 

N 

5000 

25.0% 

quartile 

2.0115 

10.0% 

1.3612 

2.5% 

1.0370 

0.5% 

1.0343 

0.0% 

minimum 

0.9336 

Figure  34.  Distribution  of  avgTimeAtWFForResRecRequests 


b.  Regression  Model  Built  by  Using  Only  Main  Effects 


After  performing  stepwise  regression  analysis  by  using  only  the  main 
effects  of  27  input  factors,  a  regression  model  for  the  avgTimeAtWFForResRecRequests 
was  built.  Figure  35  shows  the  actual  by  predicted  plot  of  this  model.  The  p- value  of  this 
model  is  less  than  0.0001,  which  means  the  model  is  statistically  significant.  The  R 
value  of  0.89  indicates  that  89%  of  the  variability  in  the  response  variable  is  explained  by 
the  model. 
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Figure  35.  Actual  by  Predicted  Plot  of  Model  for  avgTimeAtWFForResRecRequests 


Figure  36  shows  Summary  of  Fit  (SoF)  and  Analysis  of  Variance  (AoV) 
tables  of  the  regression  model. 


Summary  of  Fit 


RSquare 

0.009662 

RSquare  AdJ 

0.009300 

Root  Mean  Square  Error 

0.294611 

Mean  of  Response 

2.64015 

Observations  [or  Sum  Wgts) 

5000 

Analysis  of  Variance 

Sum  of 


Source 

DF 

Squares 

Mean  Square 

F  Ratio 

Model 

16 

3407.2975 

217.956 

2511.130 

Error 

4903 

432.5032 

0.007 

Prob>  F 

G.  Total 

4999 

3919.0007 

0.0000^ 

Figure  36.  SoF  and  AoV  Tables  of  Model  for  avgTimeAtWFForResRecRequests 


As  seen  from  Figure  37,  there  are  sixteen  statistically  significant  terms  in 
the  model  sorted  by  their  importance  on  response. 
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Sorted  Parameter  Estimates 


Term 

modeNumReq 

aCDefProb 

modeHighPriDueDays 

pilotFilErProb 

opImgTProb 

badWeatConProb 

tlCamsDefProb 

modeACRepTime 

tSCamsDefProb 

tSCamsDefProb 

t2CamsDefProb 

t4CamsDefProb 

numACs 

numTSCams 

p  ri  D  u  e  D  ays  Lo  wO  ve  r  H  i  g  h  Ratio 
camsRepTimeStraOverT  acRatio 


Estimate 

0.2798772 
5.650032 
0.32994 
3.9362399 
4.7570963 
3.0027538 
1.0819422 
0.0002001 
1 .0552377 
0.8492586 
0.7443508 
0.924482 
-0.041932 
-0.0415 
0.4822559 
-0.430398 


Std  Error 

t  Ratio 

0.002094 

133.65 

0.071454 

79.07 

0.005129 

64.33 

0.071454 

55.09 

0.095277 

49.93 

0.071453 

42.02 

0.035726 

30.28 

6.61 6e-6 

30.25 

0.035732 

29.53 

0.035728 

23.77 

0.035729 

20.83 

0.047635 

19.41 

0.00242 

-17.33 

0.003007 

-13.80 

0.035731 

13.50 

0.035737 

-12.04 

Prob>|t| 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 


Figure  37.  Sorted  Parameter  Estimates  of  Model  for  avgTimeAtWFForResReeRequests 

As  seen  from  Figure  37,  eoeffieients  for  mode  number  of  requests,  aireraft 

defeet  probability,  mode  of  high  priority  reeonnaissanee  requests’  due  days,  pilot  filming 

error  probability,  optie  image  type  reeonnaissanee  requests  probability,  bad  weather 

eondition  probability,  all  of  the  eameras’  defeet  probabilities,  mode  of  aireraft  repair  time 

and  due  days  ratio  of  low  over  high  priority  reeonnaissanee  requests  are  all  positive, 

whieh  makes  sense.  The  positive  effeets  of  a  few  terms  on 

avgTimeAtWFForResReeRequests  — namely,  the  mode  of  high  priority  reeonnaissanee 

requests’  due  days,  optie  image  type  reeonnaissanee  requests  probability,  and  due  days 

ratio  of  low  over  high  priority  reeonnaissanee  requests — may  seem  eounter-intuitive.  As 

the  mode  of  high  priority  reeonnaissanee  requests’  due  days  and  due  days’  ratio  of  low 

over  high  priority  reeonnaissanee  requests  inerease,  arrived  but  not  responded 

reeonnaissanee  requests  will  wait  in  resourees  in  the  reeonnaissanee  workflow  for  long 

time.  So  this  effeet  will  inerease  avgTimeAtWFForResReeRequests.  As  optie  image  type 

reeonnaissanee  requests  probability  inereases,  reeonnaissanee  requests  waiting  for 

daytime  missions  will  inerease  and  requests  waiting  for  night  missions  will  deerease. 

There  is  not  enough  time  for  aeeomplishing  all  daytime  missions  in  a  day.  So  arrived  but 

not  responded  reeonnaissanee  requests  requiring  daylight  missions  will  wait  in  the 
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reconnaissance  workflow  for  long  time,  and  this  effect  will  increase 
avgTimeAtWFForResRecRequests.  Coefficients  for  the  number  of  aircraft,  the  number 
of  type  5  cameras  (single  strategic  camera),  and  the  ratio  of  strategic  over  tactic  cameras’ 
repair  time  are  all  negative,  which  also  makes  sense.  The  ratio  of  strategic  over  tactic 
cameras’  repair  time’s  negative  effect  on  avgTimeAtWFForResRecRequests  seems 
counter-intuitive.  But  as  it  increases,  the  number  of  reconnaissance  requests  waiting  for 
resources  decreases  and  number  of  removed  reconnaissance  requests  increases.  Our  MoE 
deals  with  only  responded  reconnaissance  requests;  avgTimeAtWFForResRecRequests 
decreases  too. 


Residual  by  Predicted  Plot 


1  2  3  4  5 

avgTimeAtWFForResRecRequests 
Predicted 


Figure  38.  Residual  by  Predicted  Plot  of  Model  for  avgTimeAtWFForResRecRequests 
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Figure  39.  Residuals  of  Model  for  avgTimeAtWFForResRecRequests 
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As  seen  from  Figure  38,  the  vertical  spread  of  the  residuals  is  not  the 
same.  Even  though  of  the  model  is  fairly  high,  a  model  that  includes  some  interactions 
or  quadratic  effects  may  provide  a  better  fit.  As  seen  from  Figure  39,  the  distribution  of 
residuals  is  almost  symmetric  and  unimodal,  which  is  good. 

c.  Regression  Model  Built  by  Using  Main  Effects,  Interactions  and 
Quadratic  Effects 

After  making  a  stepwise  regression  analysis  by  using  main  effects, 
interactions  and  quadratic  effects  of  twenty-seven  input  factors,  a  regression  model  for 
the  avgXimeAtWFForResRecRequests  was  built.  Figure  40  shows  the  actual  by  predicted 
plot  of  this  model.  The  p-value  of  this  model  is  less  than  0.0001,  which  means  the  model 
is  statistically  significant.  The  R^  value  of  0.91  indicates  that  91%  of  the  variability  in  the 
response  variable  is  explained  by  the  model,  which  is  very  good. 


Figure  40.  Actual  by  Predicted  Plot  of  Model  for  avgTimeAtWFForResRecRequests 

Figure  41  shows  Summary  of  Fit  (SoF)  and  Analysis  of  Variance  (AoV) 
tables  of  the  regression  model. 
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Summary  of  Fit 

RSquare  0.90SB17 

RSquareAdj  0.903506 

Root  fslean  Square  Error  0.267347 

Mean  of  Response  2.64015 

Observations  {or  Sum  Wgts)  5000 


Analysis  of  Variance 


Sum  of 


Source 

DF 

Squares 

Mean  Square 

F  Ratio 

Model 

17 

3562.3818 

209.552 

2920.907 

Error 

4902 

357.4189 

0.072 

Prob>  F 

G.  Total 

4999 

3919.8007 

0.0000^ 

Figure  4 1 .  SoF  and  AoV  Tables  of  Model  for  avgTimeAtWFForResRecRequests 

As  seen  from  Figure  42,  there  are  seventeen  statistically  significant  terms 
in  the  model,  sorted  by  their  importance  on  response. 


Sorted  Parameter  Estimates 


Term 

modeNumReq 

aCDefProb 

modeHighPriDueDays 

pilotFilErProb 

opImgTProb 

badWeatConProb 

tlCamsDefProb 

tSCamsDefProb 

(t1CannsDefProb-0.35)*(nnodeACRepTinne-2520) 

modeACRepTime 

(aCDefProb-0.2)*(nnodeACRepTinne-2520) 

tSCamsDefProb 

t4CannsDefProb 

numACs 

(oplnngTProb-0.825)*(badWeatConProb-0.2) 

(nnodeNunnReq-12.98)*(aCDefProb-0.2) 

(t3CannsDefProb-0.35)*(t4CannsDefProb-0.35) 


Estimate 

0.2767029 

5.2186091 

0.3512534 

3.9505219 

4.2817056 

2.9759199 

1.3161107 

1.2499251 

-0.001598 

0.0001823 

0.0030971 

0.8118633 

0.9265566 

-0.041932 

-23.32349 

-0.253204 

1.7470647 


Std  Error 

0.001957 

0.067719 

0.004789 

0.065683 

0.088198 

0.067793 

0.033542 

0.032901 

5.359e-5 

6.192e-6 

0.000107 

0.03312 

0.046538 

0.0022 

1.535017 

0.036634 

0.410275 


t  Ratio 


Prob>|t| 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 


Figure  42.  Sorted  Parameter  Estimates  of  Model  for  avgTimeAtWFForResRecRequests 

A  discussion  about  the  sign  and  importance  of  main  effects  was  provided 
in  the  previous  section.  As  seen  from  this  model,  in  addition  to  main  effects,  there  are 
five  interaction  terms.  Figure  43  shows  the  interaction  profiler. 
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Interaction  Profiles 
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Figure  43.  Interaction  Profiler  of  Model  for  avgXimeAtWFForResRecRequests 

The  first  remarkable  interaction  is  the  one  between  type  1  camera  defect 
probability  and  mode  of  aircraft  repair  time’s  distribution.  For  the  type  1  camera  defect 
probability  of  0.55,  an  increase  in  the  mode  of  aircraft  repair  time  decreases  the 
avgXimeAtWFForResRecRequests  a  little,  which  is  counterintuitive.  The  reason  behind 
this  effect  is  a  decrease  in  the  number  of  responded  reconnaissance  requests.  However, 
for  the  type  1  camera  defect  probability  of  0.15,  an  increase  in  the  mode  of  aircraft  repair 
time  increases  the  avgXimeAtWFForResRecRequests  a  lot,  which  is  intuitive.  The  reason 
behind  this  big  increase  in  the  avgXimeAtWFForResRecRequests  is  the  need  for  more 
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aircraft  (because  we  have  more  available  cameras  with  this  small  camera  defect 
probability)  to  satisfy  more  reconnaissance  requests  waiting  in  the  queue. 

Another  interesting  interaction  is  the  one  between  aircraft  defect 
probability  and  mode  of  aircraft  repair  time’s  distribution.  For  the  aircraft  defect 
probability  of  0.1,  an  increase  in  the  mode  of  aircraft  repair  time  changes  the 
avgTimeAtWFForResRecRequests  a  little.  However,  for  the  aircraft  defect  probability  of 
0.3,  an  increase  in  the  mode  of  aircraft  repair  time  increases  the 
avgTimeAtWFForResRecRequests  a  lot.  We  can  conclude  that  we  have  a  redundant 
number  of  aircrafts  and  we  see  the  effect  of  aircraft  repair  time  when  the  aircraft  defect 
probability  increases  to  0.3. 


Residual  by  Predicted  Plot 


1  2  3  4  5 

avgTimeAtWFForResRecRequests 
Predicted 


Figure  44.  Residual  by  Predicted  Plot  of  Model  for  avgTimeAtWFForResRecRequests 

As  seen  from  Figure  44,  the  vertical  spread  of  the  residuals  is  not  the 
same.  Even  though  the  R  of  the  model  is  fairly  high,  a  model  that  includes  more  terms 
may  provide  a  better  fit.  As  seen  from  Figure  45,  the  distribution  of  residuals  is  almost 
symmetric  and  unimodal,  which  is  good. 
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Figure  45.  Residuals  of  Model  for  avgTimeAtWFForResRecRequests 
d.  Partition  Tree  Model 

Another  model  analysts  may  find  useful  is  the  partition  tree  model.  The 
partition  tree  is  a  nonparametric  tool  “that  recursively  partitions  the  data  to  provide  the 
most  explanatory  power  for  a  performance  of  interest”  (Kleijnen  et  ah,  2005).  For  some 
data  sets,  regression  models  and  partition  trees  will  have  similar  explanatory  power.  For 
other  data  sets,  one  type  of  model  may  more  clearly  capture  the  relationships  between  the 
input  factors  and  the  response.  One  benefit  of  partition  trees  is  that  they  can  be  easier  to 
explain  to  a  decision  maker,  who  may  want  to  focus  on  a  few  key  inputs  that  have 
substantial  impacts  on  the  MoE. 
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Figure  46.  Partition  Tree  Model  for  avgTimeAtWFForResReeRequests 


Figure  46  shows  a  partition  tree  example  with  only  five  splits.  The  R^ 
value  of  0.553  indicates  that  55%  of  the  variability  in  the  response  variable  is  explained 
by  the  model.  More  splits  could  be  added  to  this  tree  if  future  analysts  wish  to  examine 
smaller  subsets  of  the  data  in  more  detail  and  to  obtain  more  explanatory  power  from 
model.  The  particular  areas  of  interest  in  the  tree  are  the  leftmost  and  rightmost  areas.  In 
the  leftmost  area  we  see  that  when  the  mode  number  of  requests  arriving  to  squadron  is 
less  than  thirteen,  the  avgTimeAtWFForResReeRequests  is  low  (Mean  2.05  with  a  SD 
0.64),  which  makes  sense.  In  the  rightmost  area  we  see  that  when  the  mode  number  of 
requests  arriving  to  squadron  is  greater  than  or  equal  to  thirteen,  the  mode  number  of  due 
days  for  high  priority  reconnaissance  requests  is  greater  than  or  equal  to  six,  half  distance 
of  triangular  distribution  for  repair  time  of  tactical  cameras  (all  cameras  except  type  5)  is 
greater  than  or  equal  to  forty-six,  optic  image  type  probability  for  reconnaissance 
requests  is  greater  than  or  equal  to  0.89  and  type  4  cameras’  (used  only  for  night 

reconnaissance  missions)  defect  probability  is  less  than  0.3,  then  the 
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avgTimeAtWFForResRecRequests  is  much  higher  (Mean  4.86  with  a  SD  0.09),  whieh 
also  makes  sense.  Type  4  eameras’  defect  probability’s  positive  effeet  on 
avgTimeAtWFForResRecRequests  seems  counter-intuitive.  But  as  it  decreases,  the 
number  of  responded  reconnaissanee  requests  inereases  because  of  an  inereasing  number 
of  sueeessful  night  missions.  Our  MoE  deals  with  only  responded  reeonnaissance 
requests,  so  avgTimeAtWFForResRecRequests  increases,  too. 

2.  Removed  Over  Arrived  Reconnaissance  Requests  Ratio 


a.  Basic  Statistics 


Figure  47  shows  the  basic  statistics  for  the  remOverArrReeRequestsRatio 
(removed  over  arrived  reeonnaissanee  requests  ratio).  With  the  realistie  values  that  the 
authors  used,  the  mean  of  removed  over  arrived  reconnaissanee  requests  ratio  is  0.092 
with  a  standard  deviation  of  0.08.  Distribution  of  remOverArrReeRequestsRatio  is 
skewed  to  the  left  and  not  unimodal. 
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Figure  47.  Distribution  of  remOverArrReeRequestsRatio 


b.  Regression  Model  Built  by  Using  Only  Main  Effects 


After  using  a  stepwise  regression  analysis  with  only  the  main  effeets  of 
twenty-seven  input  faetors,  a  regression  model  for  the  remOverArrReeRequestsRatio  was 
built.  Figure  48  shows  the  aetual  by  predicted  plot  of  this  model.  The  p-value  of  this 
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model  is  less  than  0.0001,  whieh  means  the  model  is  statistieally  signifieant.  The  R 
value  of  0.86  indicates  that  86%  of  the  variability  in  the  response  variable  is  explained  by 
the  model. 


Figure  48.  Actual  by  Predicted  Plot  of  Model  for  remOverArrRecRequestsRatio 


Figure  49  shows  the  corresponding  Summary  of  Fit  (SoF)  and  Analysis  of 
Variance  (AoV)  tables  for  this  regression  model. 
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Figure  49.  SoF  and  AoV  Tables  of  Model  for  remOverArrRecRequestsRatio 


As  seen  from  Figure  50,  there  are  eighteen  statistically  significant  terms, 
sorted  by  their  importance  on  response. 
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Sorted  Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 
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Figure  50.  Sorted  Parameter  Estimates  of  Model  for  remOverArrReeRequestsRatio 

As  seen  from  Figure  50,  eoeffieients  for  the  mode  number  of  requests, 
aireraft  defeet  probability,  pilot  filming  error  probability,  bad  weather  eondition 
probability,  optie  image  type  reeonnaissanee  requests  probability,  all  of  the  eameras’ 
defeet  probabilities,  mode  of  aireraft  repair  time,  mode  and  half  dist  of  taetie  eameras 
repair  time  are  all  positive,  whieh  makes  sense.  Coeffieients  for  mode  and  half  distanee 
of  high  priority  due  days  distribution,  number  of  airerafts,  due  days  ratio  of  low  over  high 
priority  reeonnaissanee  requests  and  number  of  type  5  eameras  (single  strategie  eamera) 
are  all  negative,  whieh  also  makes  sense. 
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Residual  by  Predicted  Plot 
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Figure  5 1 .  Residual  by  Predicted  Plot  of  Model  for  remOverArrRecRequestsRatio 


Residual  remOverArrRecRequestsRatio 


Figure  52.  Residuals  of  Model  for  remOverArrRecRequestsRatio 

As  seen  from  Figure  51,  the  vertical  spread  of  the  residuals  is  not  same, 
and  curvature  is  evident.  Even  though  the  R  of  the  model  is  fairly  high,  a  model  that 
includes  some  interactions  or  quadratic  effects  may  provide  a  better  fit.  As  seen  from 
Figure  52,  the  distribution  of  residuals  is  unimodal  but  skewed  right. 

c.  Regression  Model  Built  by  Using  Main  Effects,  Interactions  and 
Quadratic  Effects 

After  using  a  stepwise  regression  analysis  with  main  effects,  interactions 
and  quadratic  effects  of  twenty-seven  input  factors,  a  regression  model  for 
remOverArrRecRequestsRatio  was  built.  Figure  53  shows  the  actual  by  predicted  plot  of 
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this  model.  The  p-value  of  this  model  is  less  than  0.0001,  whieh  means  the  model  is 
statistieally  signifieant.  The  value  of  0.91  indicates  that  91%  of  the  variability  in  the 
response  variable  is  explained  by  the  model,  which  is  very  good. 


Figure  53.  Actual  by  Predicted  Plot  of  Model  for  remOverArrRecRequestsRatio 


Figure  54  shows  the  Summary  of  Fit  (SoF)  and  Analysis  of  Variance 
(AoV)  tables  for  this  regression  model. 
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Figure  54.  SoF  and  AoV  Tables  of  Model  for  remOverArrRecRequestsRatio 


As  seen  from  Figure  55,  there  are  eighteen  statistically  significant  terms  in 
the  model,  sorted  by  their  importance  on  response. 
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Sorted  Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 
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Figure  55.  Sorted  Parameter  Estimates  of  Model  for  remOverArrRecRequestsRatio 

A  discussion  about  sign  and  importance  of  main  effects  was  provided  in 
the  previous  section.  As  seen  from  this  model,  in  addition  to  main  effects,  there  are  four 
interaction  terms  and  one  quadratic  term.  Figure  56  shows  the  interaction  profiler. 
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Figure  56.  Interaction  Profiler  of  Model  for  remOverArrRecRequestsRatio 


The  first  remarkable  interaction  is  the  one  between  type  2  camera  defect 
probability  and  mode  of  aircraft  repair  time’s  distribution.  For  the  type  2  camera  defect 
probability  of  0.55,  an  increase  in  the  mode  of  aircraft  repair  time  increases  the 
remOverArrRecRequestsRatio  a  little.  However,  for  the  type  2  camera  defect  probability 
of  0.15,  an  increase  in  the  mode  of  aircraft  repair  time  increases  the 
remOverArrRecRequestsRatio  a  lot.  The  reason  behind  this  relatively  big  increase  in  the 
remOverArrRecRequestsRatio  is  the  need  for  more  aircraft  (because  we  have  more 
available  cameras  with  this  small  camera  defect  probability)  to  satisfy  more 
reconnaissance  requests  waiting  in  the  queue. 

Another  interesting  interaction  is  the  one  between  aircraft  defect 
probability  and  mode  of  aircraft  repair  time’s  distribution.  For  the  aircraft  defect 
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probability  of  0.1,  an  increase  in  the  mode  of  aireraft  repair  time  ehanges  the 
remOverArrRecRequestsRatio  a  little.  However,  for  the  aircraft  defect  probability  of  0.3, 
an  increase  in  the  mode  of  aircraft  repair  time  inereases  the 
remOverArrRecRequestsRatio  a  lot.  We  ean  conelude  that  we  have  redundant  number  of 
airerafts  and  we  see  the  effeet  of  aireraft  repair  time  when  the  aireraft  defeet  probability 
inereases  to  0.3. 

Another  noteworthy  interaetion  is  the  one  between  due  days  ratio  of  low 
over  high  priority  reeonnaissanee  requests  and  type  3  camera  defeet  probability.  For  the 
1.6  due  days  ratio  of  low  over  high  priority  reeonnaissanee  requests  (low  priority 
reeonnaissanee  requests  can  wait  more  in  the  system  before  their  removal  from  queue  or 
provided  reeonnaissanee  report),  an  increase  in  the  type  3  camera  defect  probability 
(small  number  of  available  type  3  cameras)  deereases  the  remOverArrReeRequestsRatio 
a  little,  which  is  counterintuitive.  We  ean  conclude  that  we  have  redundant  number  of 
cameras  with  the  same  properties  of  type  3  eamera.  However,  for  the  1 .2  due  days  ratio 
of  low  over  high  priority  reeonnaissanee  requests,  an  increase  in  the  type  3  eamera  defect 
probability  increases  the  remOverArrReeRequestsRatio,  whieh  is  intuitive. 


Residual  by  Predicted  Plot 


remOverArrRecRequestsRatio 
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Figure  57.  Residual  by  Predicted  Plot  of  Model  for  remOverArrRecRequestsRatio 


As  seen  from  Figure  57,  the  vertieal  spread  of  the  residuals  is  not  same. 

Even  though  the  R^  of  the  model  is  fairly  high  and  the  eurvature  has  been  redueed 
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somewhat,  a  model  that  includes  more  terms  may  provide  a  better  fit.  As  seen  from 
Figure  58,  distribution  of  residuals  is  unimodal  but  skewed  right  a  little. 


Figure  58.  Residuals  of  Model  for  remOverArrRecRequestsRatio 

d.  Partition  Tree  Model 

Figure  59  shows  a  partition  tree  example  with  only  five  splits.  The  R 
value  of  0.435  indicates  that  44%  of  the  variability  in  the  response  variable  is  explained 
by  the  model.  More  splits  could  be  added  to  this  tree  if  future  analysts  wish  to  examine 
smaller  subsets  of  the  data  in  more  detail  and  to  obtain  more  explanatory  power  from 
model.  The  particular  areas  of  interest  in  the  tree  are  the  leftmost  and  rightmost  areas.  In 
the  leftmost  area  we  see  that  when  the  mode  number  of  requests  arriving  to  squadron  is 
less  than  twelve,  type  4  cameras’  (used  only  for  night  reconnaissance  missions)  defect 
probability  is  less  than  0.5,  bad  weather  condition  probability  is  less  than  0.25,  and  mode 
of  triangular  distribution  for  repair  time  of  tactical  cameras  (all  cameras  except  type  5)  is 
greater  than  or  equal  to  ninety-four,  remOverArrRecRequestsRatio  is  low  (Mean  0.008 
with  a  SD  0.009).  Except  for  the  mode  of  triangular  distribution  for  repair  time  of  tactical 
cameras,  all  other  factors’  negative  effects  on  remOverArrRecRequestsRatio  make  sense. 
In  the  rightmost  area  we  see  that  when  the  mode  number  of  requests  arriving  to  squadron 
is  greater  than  or  equal  to  twelve  and  aircraft  defect  probability  is  greater  than  or  equal  to 
0.3,  remOverArrRecRequestsRatio  is  high  (Mean  0.34  with  a  SD  0.09),  which  makes 
sense. 
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Figure  59.  Partition  Tree  Model  for  remOverArrRecRequestsRatio 


3.  Robust  Configuration  for  Reconnaissance  Squadron 

After  using  a  stepwise  regression  analysis  with  main  effects,  interactions  and 
quadratic  effects  of  eight  decision  factors  regression  models  for  mean  and  standard 
deviations  of  remOverArrRecRequestsRatio  and  avgTimeAtWFForResRecRequests  were 
built.  The  remOverArrRecRequestsRatio  MoE  is  more  important  than  the 

avgTimeAtWFForResRecRequests  MoE  for  the  reconnaissance  squadron  commander. 
Thus,  we  will  deal  with  models  for  remOverArrRecRequestsRatio  first.  If  there  will  be 
any  conflicting  decision  factor  setting(s),  the  one(s)  used  for 

remOverArrRecRequestsRatio  will  be  recommended.  Figure  60  shows  the  Prediction 
Profiler  for  Model  of  remOverArrRecRequestsRatio ’s  Standard  Deviation.  Relevant 
decision  factors  were  set  to  minimize  the  standard  deviation  of 
remOverArrRecRequestsRatio,  and  these  same  values  were  used  to  minimize  mean  of 
remOverArrRecRequestsRatio,  as  seen  in  Figure  61. 
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Prediction  Profiier 


8  10  6  25 

numType2Cams  numTypeSCams  numType5Cams  numACs 


Figure  60.  Prediction  Profiler  for  Model  of  remOverArrRecRequestsRatio’s  Standard 

Deviation 
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Figure  61 .  Prediction  Profiler  for  Model  of  remOverArrRecRequestsRatio’s  Mean 

Figure  62  shows  the  Prediction  Profiler  for  Model  of 
avgXimeAtWFForResRecRequests’s  Standard  Deviation.  Relevant  decision  factors  were 
set  to  minimize  standard  deviation  of  avgTimeAtWFForResRecRequests,  and  these  same 
values  were  used  to  minimize  mean  of  avgTimeAtWFForResRecRequests,  as  seen  in 
Figure  63. 


90 


CMCO'^LOCDI^OOOCN  CO  UO  CDOO'^CNCO'^LOCD 

t-CNCNCNCvICvICvICN 


8  2  25 
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Figure  62.  Prediction  Profiler  for  Model  of  avgTimeAtWFForResRecRequests’s 

Standard  Deviation 
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Figure  63.  Prediction  Profiler  for  Model  of  avgTimeAtWFForResRecRequests’s  Mean 

After  comparing  decision  factors’  settings  for  remOverArrRecRequestsRatio  and 
avgTimeAtWFForResRecRequests  MoEs,  we  see  that  there  is  only  one  conflicting 
decision  factor  setting,  which  is  numTypeSCams.  It  was  set  to  six  to  minimize  the 
standard  deviation  of  remOverArrRecRequestsRatio,  and  this  value  is  recommended.  The 
decision  factors  not  seen  in  either  Prediction  Profilers  for  remOverArrRecRequestsRatio 
or  avgTimeAtWFForResRecRequests  are  set  to  the  minimum  levels  used  in  experiment. 
Table  4  shows  the  robust  configuration  for  the  reconnaissance  squadron  in  configuration: 
RF-4  /  situation:  peace  scenario. 
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Number 

Decision  Factors 

Vaiue 

1 

nuniTlCam 

10 

2 

nLimT2Can'i 

8 

3 

numTSCani 

10 

4 

numT4Cam 

8 

5 

nuniTBCam 

6 

6 

numACs 

25 

7 

nuniPilots 

35 

8 

numAnalysts 

5 

Table  4.  Robust  Configuration  for  Reconnaissance  Squadron  in  Configuration:  RF- 

4  /  Situation:  Peace  Scenario 


D.  OUTPUT  ANALYSIS  FOR  CONFIGURATION:  RF-4  /  SITUATION:  WAR 


I.  Removed  Over  Arrived  Reconnaissance  Requests  Ratio 


a.  Basic  Statistics 


Figure  64  shows  the  basic  statistics  for  the  remOverArrRecRequestsRatio 
(removed  over  arrived  reconnaissance  requests  ratio).  With  the  realistic  values  the 
authors  used,  the  mean  of  removed  over  arrived  reconnaissance  requests  ratio  is  0.61  with 
a  standard  deviation  of  0.17.  The  distribution  of  remOverArrRecRequestsRatio  is  skewed 
to  the  right  and  is  unimodal. 


r  e  mo  ved  Over  Arr  ive  d  Rec  Req  uestsRati  o 


Quantiles  Moments 


100.0% 

maximum 

0.89183 

Mean 

0.615133 

99.5% 

0.87596 

Std  Dev 

0.1720218 

97.5% 

0.85563 

Std  Err  Mean 

0.0024328 

90.0% 

0.8284 

Upper  95%  Mean 

0.6199023 

75.0% 

quartile 

0.7702 

Lower  95%  Mean 

0.6103638 

50.0% 

median 

0.63517 

N 

5000 

25.0% 

quartile 

0.48571 

10.0% 

0.36881 

2-5% 

0.27077 

0.5% 

0.18048 

0.0% 

minimum 

0.09778 

Figure  64.  Distribution  of  remOverArrRecRequestsRatio 
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b.  Regression  Model  Built  by  Using  Only  Main  Effects 


After  using  a  stepwise  regression  analysis  with  only  the  main  effects  of 
twenty-five  input  factors,  a  regression  model  for  the  remOverArrRecRequestsRatio  was 
built.  Figure  65  shows  the  actual  by  predicted  plot  of  this  model.  The  p- value  of  this 
model  is  less  than  0.0001,  which  means  the  model  is  statistically  significant.  The  R 
value  of  0.37  indicates  that  37%  of  the  variability  in  the  response  variable  is  explained  by 
the  model. 


Actual  by  Predicted  Plot 


'  I  ’ ' '  I " '  I ' "  I  "  '  I " '  I ' "  I  "  '  I " '  I ' 

0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9 
removedOverArrRecRequestsRatio 
Predicted  P<.0001  RSq=0.37 
RMSE=0.1363 


Figure  65.  Actual  by  Predicted  Plot  of  Model  for  remOverArrRecRequestsRatio 


Figure  66  shows  the  Summary  of  Fit  (SoF)  and  Analysis  of  Variance 
(AoV)  tables  of  the  regression  model  built  for  remOverArrRecRequestsRatio. 


Summary  of  Fit 

RSquare  0.374913 

RSquare  Adj  0.372654 

Root  Mean  Square  Error  0.13625 

Mean  of  Response  0.615133 

Observations  (or  Sum  Wgts)  5000 

Analysis  of  Variance 


Sum  of 


Source 

DF 

Squares 

Mean  Square 

F  Ratio 

Model 

18 

55.46013 

3.08112 

165.9719 

Error 

4981 

92.46777 

0.01856 

Prob  >  F 

C.  Total 

4999 

147.92790 

0.0000* 

Figure  66.  SoF  and  AoV  Tables  of  Model  for  remOverArrRecRequestsRatio 
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Figure  67  shows  eighteen  statistically  significant  terms,  sorted  by  their 
importance  on  response. 


Sorted  Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

aCDefProb 

0.8101601 

0.033043 

24.52 

numACs 

-0.026452 

0.001119 

-23.63 

modeNumReq 

0.0178984 

0.000961 

18.62 

numT3Cams 

0.0170652 

0.000973 

17.53 

modeACRepTime 

0.0000524 

3.059e-6 

17.13 

aCInterceptProb 

0.7140438 

0.04406 

16.21 

numPilots 

0.006542 

0.000591 

11.07 

pilotFilErProb 

0.3630568 

0.033043 

10.99 

badWeatConProb 

0.3081087 

0.033043 

9.32 

numTlCams 

-0.009159 

0.001119 

-8.18 

numAnalysts 

0.0083583 

0.001141 

7.32 

numTZCams 

-0.012129 

0.001733 

-7.00 

tlCamsDefProb 

0.1147619 

0.016522 

6.95 

t4CamsDefProb 

0.0729393 

0.016521 

4.41 

t2CamsDefProb 

0.068144 

0.016522 

4.12 

t3CamsDefProb 

0.0585527 

0.016521 

3.54 

tSCamsDefProb 

0.0582391 

0.016521 

3.53 

numTSCams 

-0.004086 

0.001391 

-2.94 

Prob>|t| 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

0.0004* 

0.0004* 

0.0033* 


Figure  67.  Sorted  Parameter  Estimates  of  Model  for  remOverArrRecRequestsRatio 


As  seen  from  Figure  67,  coefficients  for  aircraft  defect  probability,  mode 
number  of  reconnaissance  requests,  number  of  type  3  cameras,  mode  time  of  aircraft 
repair,  aircraft  interception  probability,  number  of  pilots,  pilot  filming  error  probability, 
bad  weather  condition,  number  of  analysts,  type  1,2,3, 4  and  5  camera  defect  probability, 
are  all  positive.  Most  of  these  make  sense,  except  for  the  number  of  type  3  cameras, 
number  of  pilots,  and  number  of  analysts  which  seems  counter-intuitive.  Coefficients  for 
number  of  aircraft,  number  of  type  1,  2  and  5  cameras,  are  all  negative,  which  also  makes 
sense. 


94 


Residual  by  Predicted  Plot 


Figure  68.  Residual  by  Predicted  Plot  of  Model  for  remOverArrRecRequestsRatio 

Residual 

removedOverArrRecRequestsRatio 


T . . ...- 

> 
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1 

i-rlff 
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-0.3  -0.1 

0  0.1  0.2  0.3  0.4 

Figure  69.  Residuals  of  Model  for  remOverArrRecRequestsRatio 


As  seen  from  Figure  68,  the  vertical  spread  of  the  residuals  is  not  the 
same.  A  model  that  includes  some  interactions  or  quadratic  effects  may  provide  a  better 
fit.  As  seen  from  Figure  69,  distribution  of  residuals  is  unimodal  but  not  symmetric. 


c.  Regression  Model  Built  by  Using  Main  Effects,  Interactions  and 
Quadratic  Effects 

After  using  a  stepwise  regression  analysis  with  main  effects,  interactions 
and  quadratic  effects  of  twenty-five  input  factors,  a  regression  model  for  the 
remOverArrRecRquestsRatio  was  built.  Figure  70  shows  the  actual  by  predicted  plot  of 
this  model.  The  p-value  of  this  model  is  less  than  0.0001,  which  means  the  model  is 
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statistically  significant.  The  R  value  of  0.71  indicates  that  71%  of  the  variability  in  the 
response  variable  is  explained  by  the  model. 


Actual  by  Predicted  Plot 


Figure  70.  Actual  by  Predicted  Plot  of  Model  for  remOverArrReeRequestsRatio 


Figure  71  shows  the  Summary  of  Fit  (SoF)  and  Analysis  of  Varianee 
(AoV)  tables  of  the  regression  model  built  for  remOverArrReeRequestsRatio. 


Summary  of  Fit 

RSquare  0.706679 

RSquare  Adj  0.704789 

Root  Mean  Square  Error  0.093465 

Mean  of  Response  0.615133 

Observations  (or  Sum  Wgts)  5000 

Analysis  of  Variance 


Sum  of 


Source 

DF 

Squares 

Mean  Square 

F  Ratio 

Model 

32 

104.53755 

3.26680 

373.9584 

Error 

4967 

43.39035 

0.00874 

Prob  >  F 

C.  Total 

4999 

147.92790 

0.0000* 

Figure  7 1 .  SoF  and  AoV  Tables  of  Model  for  remOverArrReeRequestsRatio 


Figure  72  shows  thirty-two  significant  terms  in  the  model,  sorted  by  their 
importance  on  the  response.  A  discussion  about  the  sign  and  importance  of  main  effects 
was  provided  in  the  previous  seetion.  As  seen  from  this  model,  in  addition  to  main 
effeets,  there  are  three  quadratie  terms  and  fourteen  interaction  terms.  Figure  73  shows 
the  interaetion  profiler. 
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Sorted  Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

numACs 

-0.060318 

0.001035 

-58.27 

0.0000* 

(numT3Cams-7)*(numT3Cams-7) 

0.0303606 

0.000604 

50.28 

1 

0.0000* 

aCDefProb 

0.8103118 

0.022667 

35.75 

<.0001* 

numTlCams 

-0.026004 

0.000882 

-29.48 

1 _ 

<.0001* 

(numT5Cams-4)*(numT5Cams-4) 

-0.025783 

0.000941 

-27.39 

1 

<.0001* 

modeNumReq 

0.0178894 

0.000659 

27.13 

] 

<.0001* 

(numT4Cams-5.56)*(numT4CamS“5.56) 

-0.019203 

0.000735 

-26.14 

1 

<.0001* 

(numT2Cams-6.38)*(numT3Cams-7) 

0.021198 

0.000816 

25.98 

j 

<.0001* 

(numTlCams-12.58)*(numT3Cams-7) 

-0.01524 

0.000593 

-25.68 

[ 

<.0001* 

modeACRepTime 

5.2417e-5 

2.099e-6 

24.98 

<.0001* 

(numTlCams-12.58)*(numAnalysts-7.52) 

0.0157212 

0.000635 

24.77 

i' 

<.0001* 

(numTlCams-12.58)*(numT4Cams-5.56) 

-0.016282 

0.000684 

-23.79 

' 

<.0001* 

aCInterceptProb 

0.7158112 

0.030223 

23.68 

] 

<.0001* 

(numT2Cams-6.38)*(numT4Cams-5.56) 

0.0207525 

0.00092 

22.56 

<.0001* 

(numTlCams-12.58)*(numT5Cams-4) 

-0.018113 

0.000834 

-21.73 

<.0001* 

(numT2Cams-6.38)*(numPilots-39.96) 

-0.012926 

0.000615 

-21.01 

1^ 

<.0001* 

(numT3Cams-7)*(numT4Cams-5.56) 

0.0076772 

0.000408 

18.82 

<.0001* 

(numT2Cams-6.38)*(numACs-22.42) 

0.0172401 

0.000934 

18.46 

<.0001* 

(numT5Cams-4)*(numPilots-39.96) 

0.0085447 

0.000492 

17.38 

<.0001* 

numAnalysts 

0.019773 

0.001153 

17.15 

<.0001* 

(numT2Cams-6.38)*(numAnalysts-7.52) 

0.0176838 

0.001035 

17.09 

<.0001* 

numT2Cams 

-0.02719 

0.00162 

-16.78 

r 

<.0001* 

pilotFilErProb 

0.3633031 

0.022667 

16.03 

<.0001* 

numPilots 

0.0079505 

0.000513 

15.50 

<.0001* 

(numT3Cams-7)*(numPilots-39.96) 

-0.003847 

0.000258 

-14.94 

<.0001* 

badWeatConProb 

0.3082938 

0.022667 

13.60 

<.0001* 

numTSCams 

-0.013838 

0.001243 

-11.13 

L , 

<.0001* 

(numPilots-39.96)*(numAnalysts-7.52) 

-0.00346 

0.000339 

-10.21 

L, 

<.0001* 

tlCamsDefProb 

0.1147315 

0.011333 

10.12 

<.0001* 

numT4Cams 

0.0074224 

0.000862 

8.61 

<.0001* 

(numT4Cams-5.56)*(numACs-22.42) 

-0.005989 

0.000845 

-7.08 

r 

<.0001* 

numT3Cams 

0.0033781 

0.000789 

4.28 

1 

<.0001* 

Figure  72.  Sorted  Parameter  Estimates  of  Model  for  remOverArrRecRequestsRatio 
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Interaction  Profiles 
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Figure  73.  Interaction  Profiler  of  Model  for  remOverArrRecRequestsRatio 


An  interesting  interaction  is  the  one  between  the  number  of  type  1  cameras 
and  number  of  type  5  cameras.  For  two  type  5  cameras,  an  increase  in  the  number  of  type 
1  cameras  increases  the  remOverArrRecRequestsRatio  a  little.  However,  for  six  type  5 
cameras,  an  increase  in  type  1  cameras  decreases  the  remOverArrRecRequestsRatio  a  lot, 
which  is  intuitive.  The  reason  behind  this  is  because  with  the  type  1  camera,  vertical  and 
optic  type  requests  are  planned  and  with  type  5  cameras  oblique  and  optic  type  requests 
are  planned,  increase  in  those  cameras  reduces  the  not  responded  reconnaissance 
requests. 
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Residual  by  Predicted  Plot 


Figure  74.  Residual  by  Predicted  Plot  of  Model  for  remOverArrRecRequestsRatio 


As  seen  from  Figure  75,  the  vertical  spread  of  the  residuals  is  the  same, 
which  is  good.  As  seen  from  Figure  76,  the  distribution  of  residuals  is  symmetric  and 
unimodal,  which  is  also  good. 


Residual 

removedOverArrRecRequestsRatio 
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I  1  1  1  1  t  1  1  1  1  1  1  1  ' 

1  0.1  0.2  0.3 

Figure  75.  Residuals  of  Model  for  remOverArrRecRequestsRatio 


2.  Robust  Design  for  Reconnaissance  Squadron 

After  using  a  stepwise  regression  analysis  with  main  effects,  interactions  and 

quadratic  effects  of  eight  decision  factors,  regression  models  for  mean  and  standard 

deviations  of  remOverArrRecRequestsRatio  and  avgTimeAtWFForResRecRequests  were 

built.  We  could  not  find  any  effective  model  for  remOverArrRecRequestsRatio  MoE,  so 

we  will  only  deal  with  models  for  avgTimeAtWFForResRecRequests.  Figure  76  shows 

the  Prediction  Profiler  for  the  model  of  avgTimeAtWFForResRecRequests ’s  Standard 
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Deviation.  Relevant  decision  factors  were  set  to  minimize  standard  deviation  of 
avgTimeAtWFForResRecRequests,  and  these  same  values  were  used  to  minimize  mean 
of  avgTimeAtWFForResRecRequests  as  seen  in  Figure  77. 


(NCO-^LncDI^OOO  (N  CO  lO  CD  CnOT-CMCO-^LOCD 

■i-CMCNCNCNCMCMCM 

8  5  25 

numT4Cams  numTSCams  numACs 

Figure  76.  Prediction  Profiler  for  Model  of  avgTimeAtWFForResRecRequests’s 

Standard  Deviation 


Prediction  Profiler 


8  5  25  35 

numT4Cams  numT5Cams  numACs  numPilots 


Figure  77.  Prediction  Profder  for  Model  of  avgTimeAtWFForResRecRequests’s  Mean 

The  decision  factors  not  seen  in  Prediction  Profilers  for 
avgTimeAtWFForResRecRequests  are  set  to  their  minimum  levels  used  in  experiment. 
Table  5  shows  the  robust  configuration  for  the  reconnaissance  squadron  in  configuration: 
RF-4  /  situation:  war  scenario. 
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Number 

Decision  Factors 

Value 

1 

numTlCan-i 

10 

2 

numT2Can'i 

s 

3 

numTSCam 

4 

4 

numT4Cani 

8 

S 

nuniTBCam 

S 

6 

numACs 

25 

7 

numPilots 

35 

8 

nuniAnalysts 

5 

Table  5.  Robust  Configuration  for  Reconnaissance  Squadron  in  Configuration:  RF- 

4  /  Situation:  War  Scenario 


E.  OUTPUT  ANALYSIS  FOR  CONFIGURATION:  F-I6  /  SITUATION: 
PEACE 


I.  Average  Time  at  Workflow  for  Responded  Reconnaissance  Requests 


a.  Basic  Statistics 


Figure  78  shows  the  basic  statistics  for  the  average  time  at  workflow  for 
responded  reconnaissance  requests  (avgTimeAtWFForResRecRequests).  With  the 
realistic  values  the  authors  used,  the  mean  of  avgTimeAtWFForResRecRequests  is  1.14 
days  with  a  standard  deviation  of  0.25.  Distribution  of 
avgTimeAtWFForResRecRequests  is  unimodal  but  skewed  to  the  right. 


avgTimeAtWFForResRecRequests 


Quantiles 


100.0% 

maximum 

2.8613 

99.5% 

2.2993 

97.5% 

1.8153 

90.0% 

1.4485 

75.0% 

quartile 

1.1941 

50.0% 

median 

1.0374 

25.0% 

quartile 

0.9813 

10.0% 

0.9563 

2.5% 

0.9356 

0.5% 

0.9224 

0.0% 

minimum 

0.9125 

Moments 

1.1372702 
0.2468134 
0.0034905 
1.1441131 
1.1304274 
5000 


Mean 
Std  Dev 
Std  Err  Mean 
Upper  95%  Mean 
Lower  95%  Mean 
N 


Figure  78.  Distribution  of  avgTimeAtWFForResRecRequests 
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b.  Regression  Model  Built  by  Using  Only  Main  Effects 


After  making  stepwise  regression  analysis  by  using  only  the  main  effects 
of  twenty-three  input  factors,  a  regression  model  for  the 
avgTimeAtWFForResRecRequests  was  built.  Figure  79  shows  the  actual  by  predicted 
plot  of  this  model.  The  p-value  of  this  model  is  less  than  0.0001,  which  means  the  model 
is  statistically  significant.  The  R^  value  of  0.70  indicates  that  70%  of  the  variability  in  the 
response  variable  is  explained  by  the  model,  although  the  strong  curvature  indicates  a 
problem  with  lack-of-fit. 


Figure  79.  Actual  by  Predicted  Plot  of  Model  for  avgTimeAtWFForResRecRequests 

Figure  80  shows  the  Summary  of  Fit  (SoF)  and  Analysis  of  Variance 
(AoV)  tables  for  this  regression  model. 
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[  Summary  of  Fit 

RSquare  0.701251 

RSquare  Adj  0.700412 

Root  Mean  Square  Error  0.135092 

Mean  of  Response  1.13727 

Observations  {or  Sum  Wgts)  5000 

Analysis  of  Variance 

Sum  of 


Source 

DF 

Squares 

Mean  Square 

F  Raio 

Model 

14 

213.54733 

15.2534 

035.0031 

Error 

4905 

90.97609 

0.0102 

Prob>  F 

G.  Total 

4999 

304.52343 

0.0000* 

Figure  80.  SoF  and  AoV  Tables  of  Model  for  avgTimeAtWFForResReeRequests 


As  seen  from  Figure  81,  there  are  fourteen  statistieally  signifieant  terms  in 
the  model,  sorted  by  their  importanee  on  response. 


Sorted  Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

numT2Pods 

-0.112805 

0.001587 

-71.10 

t2PodsDefProb 

2.9362028 

0.065528 

44.81 

modePodsRepTime 

0.00012 

3.034e-6 

39.56 

modeNumReq 

0.0341259 

0.000911 

37.46 

verAngTProb 

-0.854307 

0.032764 

-26.07 

badWeatConProb 

0.8061632 

0.032764 

24.61 

t1  PodsDefProb 

0.720938 

0.065527 

11.00 

modeHighPriDueDays 

0.0240071 

0.002334 

10.28 

opImgTProb 

-0.213858 

0.032766 

-6.53 

modeACRepTime 

-1 .536e-5 

3.034e-6 

-5.06 

numPilots 

0.0027387 

0.000588 

4.66 

numTIPods 

-0.0054 

0.001265 

-4.27 

dataLinkDefProb 

0.125024 

0.032763 

3.82 

numAnalysts 

-0.003696 

0.001121 

-3.30 

Prob>|t| 

0.0000* 

0.0000* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

<.0001* 

0.0001* 

0.0010* 


Figure  8 1 .  Sorted  Parameter  Estimates  of  Model  for  avgTimeAtWFForResReeRequests 

As  seen  from  Figure  81,  eoeffieients  for  Type  2  pods’  (eameras)  defeet 
probability,  mode  of  pod  repair  time,  mode  number  of  requests,  bad  weather  eondition 
probability.  Type  1  pods’  defeet  probability,  mode  of  high  priority  reeonnaissanee 
requests’  due  days,  number  of  pilots  and  data  link  eommunieation  defeet  probability  are 
all  positive,  whieh  makes  sense.  The  number  of  pilots’  positive  effeet  on 
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avgTimeAtWFForResRecRequests  is  counter-intuitive.  Our  explanation  is  that  as  the 
number  of  pilots  increase,  reconnaissance  requests  that  have  been  waiting  for  long  time  in 
the  queue  are  responded  to,  so  their  high  timeAtWF  (time  at  workflow)  will  increase  the 
avgTimeAtWFForResRecRequests.  Coefficients  for  the  number  of  Type  2  pods,  vertical 
angle  type  reconnaissance  requests  probability,  optic  image  type  reconnaissance  requests 
probability,  mode  of  aircraft  repair  time,  number  of  type  1  pods  (only  used  to  satisfy 
vertical  image  type  reconnaissance  requests)  and  number  of  analysts  are  all  negative, 
which  also  makes  sense.  The  mode  of  aircraft  repair  time’s  negative  effect  on 
avgTimeAtWFForResRecRequests  is  counter-intuitive.  As  it  increases,  the  number  of 
available  aircraft  decreases,  so  reconnaissance  requests  which  have  already  been  waiting 
for  a  long  time  in  the  queue  are  not  responded  to.  Therefore, 
avgT ime AtWFF orResRecRequests  decreases . 


Figure  82.  Residual  by  Predicted  Plot  of  Model  for  avgTimeAtWFForResRecRequests 
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Figure  83.  Residuals  of  Model  for  avgXimeAtWFForResReeRequests 

As  seen  from  Figure  82,  the  vertieal  spread  of  the  residuals  is  not  same 
and  there  strong  eurvature  is  evident.  A  model  that  ineludes  some  interaetions  or 
quadratie  effeets  may  provide  a  better  fit.  As  seen  from  Figure  83,  the  distribution  of 
residuals  is  unimodal  but  skewed  to  the  right. 

c.  Regression  Model  Built  by  Using  Main  Effects,  Interactions  and 
Quadratic  Effects 

After  making  a  stepwise  regression  analysis  by  using  main  effects, 
interactions  and  quadratic  effects  of  twenty-three  input  factors,  a  regression  model  for  the 
avgTimeAtWFForResRecRequests  was  built.  Figure  84  shows  the  actual  by  predicted 
plot  of  this  model.  The  p-value  of  this  model  is  less  than  0.0001,  which  means  the  model 
is  statistically  significant.  The  R  value  of  0.90  indicates  that  90%  of  the  variability  in  the 
response  variable  is  explained  by  the  model,  which  is  very  good. 
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Figure  84.  Actual  by  Predicted  Plot  of  Model  for  avgTimcAtWFForResRecRequests 


Figure  85  shows  the  Summary  of  Fit  (SoF)  and  Analysis  of  Variance 
(AoV)  tables  for  this  regression  model. 


Summary  of  Fit 


RSquare 

0.903697 

RSquare  Adj 

0.903349 

Root  Mean  Square  Error 

0.076731 

Mean  of  Response 

1.13727 

Observations  {or  Sum  Wgts) 

5000 

Analysis  of  Variance 

Sum  of 


Source 

DF 

Squares 

Mean  Square 

F  Fialio 

Model 

IS 

275.196S1 

15.2SS7 

2596.722 

Error 

49  SI 

29.32662 

0.0059 

Prob>  F 

C.  Total 

4999 

304.52343 

0.0000^ 

Figure  85.  SoF  and  AoV  Tables  of  Model  for  avgTimcAtWFForResRecRequests 


As  seen  from  Figure  86,  there  are  eighteen  statistically  significant  terms  in 
the  model,  sorted  by  their  importance  on  response. 
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Sorted  Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

numT2Pods 

-0.112805 

0.000901 

-125.2 

0.0000* 

t2PodsDefProb 

3.1401068 

0.038327 

81.93 

1 

0.0000* 

modePodsRepTime 

0.0001147 

1.74e-6 

65.89 

1 

0.0000* 

modeNumReq 

0.0309837 

0.000534 

58.01 

1 

0.0000* 

(numT2Pods-4.5)*(t2PodsDefProb-0. 1 ) 

-1.517096 

0.030907 

-49.09 

l- 

0.0000* 

verAngTProb 

-0.867674 

0.018786 

-46.19 

r 

0.0000* 

badWeatConProb 

0.7890755 

0.018929 

41.69 

0.0000* 

(nunnT2Pods-4.5)*(numT2Pods-4.5) 

0.0453381 

0.001108 

40.94 

J: 

<.0001* 

(numT2Pods-4.5)*(modePodsRepTime-2520) 

-0.000056 

1.431e-6 

-39.09 

1 

<.0001* 

(nnodePodsRepTime-2520)*(t2PodsDefProb-0. 1 ) 

0.0023853 

6.172e-5 

38.65 

1 

<.0001* 

(numT2Pods-4.5)*(modeNumReq-12.89) 

-0.01553 

0.00043 

-36.14 

1 

<.0001* 

(nunnT2Pods-4.5)*(verAngTProb-0.5) 

0.55645 

0.015453 

36.01 

1 

<.0001* 

(modePodsRepTime-2520)*(verAngTProb-0.5) 

-0.000761 

3.143e-5 

-24.21 

<.0001* 

t1  PodsDefProb 

0.6482204 

0.037774 

17.16 

<.0001* 

(modeNumReq-12.89)*(modePodsRepTime-2520) 

1.3076e-5 

8.527e-7 

15.34 

<.0001* 

modeHighPriDueDays 

0.0179997 

0.001353 

13.30 

<.0001* 

(verAngTProb-0.5)*(badWeatConProb-0.2) 

-3.389556 

0.323038 

-10.49 

[ 

"I 

<.0001* 

(modeHighPriDueDays-5. 1  )*(t1  PodsDefProb-0. 1 ) 

0.49652 

0.047798 

10.39 

1 

<.0001* 

Figure  86.  Sorted  Parameter  Estimates  of  Model  for  avgXimeAtWFForResRecRequests 

A  discussion  about  the  sign  and  importance  of  main  effects  was  provided 
in  the  previous  section.  As  seen  from  this  model,  in  addition  to  main  effects,  there  are 
nine  interaction  terms  and  one  quadratic  term.  Figure  87  shows  the  interaction  profiler. 
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Figure  87.  Interaction  Profiler  of  Model  for  avgTimeAtWFForResRecRequests 

One  of  the  remarkable  interactions  is  the  one  between  number  of  Type  2 
pods  and  Type  2  pods  defect  probability.  For  the  six  Type  2  pods,  an  increase  in  the  Type 
2  pods  defect  probability  increases  the  avgTimeAtWFForResRecRequests  a  little. 
However,  for  three  Type  2  pods,  an  increase  in  the  Type  2  pods  defect  probability 
increases  the  avgTimeAtWFForResRecRequests  a  lot.  We  can  conclude  that  we  need 
more  than  three  (almost  six)  Type  2  pods  to  decrease  its  negative  defect  probability  effect 
on  avgTimeAtWFForResRecRequests. 
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Figure  88.  Residual  by  Predicted  Plot  of  Model  for  avgTimeAtWFForResRecRequests 


As  seen  from  Figure  88,  the  vertical  spread  of  the  residuals  is  almost  the 
same.  As  seen  from  Figure  89,  distribution  of  residuals  is  almost  symmetric  and 
unimodal,  which  is  good. 
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Figure  89.  Residuals  of  Model  for  avgTimeAtWFForResRecRequests 
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d. 


Partition  Tree  Model 


Figure  90.  Partition  Tree  Model  for  avgTimeAtWFForResRecRequests 


Figure  90  shows  a  partition  tree  example  with  only  five  splits.  The  R^ 
value  of  0.508  indicates  that  51%  of  the  variability  in  the  response  variable  is  explained 
by  the  model.  More  splits  could  be  added  to  this  tree  if  future  analysts  wish  to  examine 
smaller  subsets  of  the  data  in  more  detail  and  to  obtain  more  explanatory  power  from 
model.  The  particular  areas  of  interest  in  the  tree  are  the  leftmost  and  rightmost  areas.  In 
the  leftmost  area  we  see  that  when  the  number  of  Type  2  pods  is  greater  than  or  equal  to 
five,  bad  weather  condition  probability  is  less  than  0.3  and  mode  number  of  requests 
arriving  to  squadron  is  less  than  thirteen,  avgTimeAtWFForResRecRequests  is  low 
(Mean  0.97  with  a  SD  0.03),  which  makes  sense.  In  the  rightmost  area  we  see  that  when 
the  number  of  Type  2  pods  is  less  than  four  and  type  2  pods’  defect  probability  is  greater 
than  0.1  avgTimeAtWFForResRecRequests  is  high  (Mean  1.5  with  a  SD  0.32),  which 
makes  sense,  too. 


2.  Removed  Over  Arrived  Reconnaissance  Requests  Ratio 


Figure  91  shows  the  basic  statistics  for  the  remOverArrRecRequestsRatio 


(removed  over  arrived  reconnaissance  requests  ratio).  With  the  realistic  values  the 
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authors  used,  the  mean  of  remOverArrRecRequestsRatio  is  0.002  with  a  standard 
deviation  of  0.007.  The  distribution  of  remOverArrRecRequestsRatio  is  skewed  to  the 
right  and  is  unimodal.  The  maximum  remOverArrRecRequestsRatio  is  0.08,  which  is 
fairly  small,  so  no  model  was  created  for  remOverArrRecRequestsRatio. 


removedOverArrivedRecRequestsRatio 
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F igure  9 1 .  Distribution  of  remOverArrRecRequestsRatio 


3.  Robust  Configuration  for  Reconnaissance  Squadron 

After  using  a  stepwise  regression  analysis  with  main  effects,  interactions  and 
quadratic  effects  of  five  decision  factors,  regression  models  for  mean  and  standard 
deviations  of  remOverArrRecRequestsRatio  and  avgTimeAtWFForResRecRequests  were 
built.  The  remOverArrRecRequestsRatio  MoE  is  more  important  than 
avgTimeAtWFForResRecRequests  MoE  for  the  reconnaissance  squadron  commander. 
Therefore  we  will  deal  with  models  for  remOverArrRecRequestsRatio  first.  If  there  will 
be  any  conflicting  decision  factor  setting(s),  the  one(s)  used  for 
remOverArrRecRequestsRatio  will  be  recommended.  Figure  92  shows  the  Prediction 
Profiler  for  the  model  of  remOverArrRecRequestsRatio ’s  standard  deviation.  Relevant 
decision  factors  were  set  to  minimize  the  standard  deviation  of 
remOverArrRecRequestsRatio,  and  these  same  values  were  used  to  minimize  the  mean  of 
remOverArrRecRequestsRatio,  as  seen  in  Figure  93. 
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Figure  92.  Prediction  Profiler  for  Model  of  remOverArrRecRequestsRatio’s  Standard 

Deviation 
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Figure  93.  Prediction  Profiler  for  Model  of  remOverArrRecRequestsRatio’s  Mean 

Figure  94  shows  the  prediction  profiler  for  the  Model  of 
avgXimeAtWFForResRecRequests’s  standard  deviation.  Relevant  decision  factors  were 
set  to  minimize  standard  deviation  of  avgTimeAtWFForResRecRequests,  and  these  same 
values  were  used  to  minimize  mean  of  avgTimeAtWFForResRecRequests,  as  seen  in 
Figure  95. 
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Figure  94.  Prediction  Profiler  for  Model  of  avgTimeAtWFForResRecRequests’s 

Standard  Deviation 
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Figure  95.  Prediction  Profiler  for  Model  of  avgTimeAtWFForResRecRequests’s  Mean 

After  comparing  decision  factors’  settings  for  remOverArrRecRequestsRatio  and 
avgTimeAtWFForResRecRequests  MoEs,  we  see  that  there  is  no  conflicting  decision 
factor.  The  decision  factors  not  seen  in  either  Prediction  Profilers  for 
remOverArrRecRequestsRatio  or  avgTimeAtWFForResRecRequests  are  set  to  their 
minimum  levels  used  in  experiment.  Table  6  shows  the  robust  configuration  for  the 
reconnaissance  squadron  in  configuration:  F-16  /  situation:  peace  scenario. 
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Number 

Decision  Factors 

Vaiue 

1 

numTlPods 

10 

2 

numT2Pod3 

6 

3 

numACs 

20 

4 

numPilots 

35 

S 

numAnalysts 

5 

Table  6.  Robust  Configuration  for  Reconnaissance  Squadron  in  Configuration:  F- 

16  /  Situation:  Peace  Scenario 


F.  OUTPUT  ANALYSIS  FOR  CONFIGURATION:  F-16  /  SITUATION:  WAR 


1.  Removed  Over  Arrived  Reconnaissance  Requests  Ratio 


a.  Basic  Statistics 


Figure  96  shows  the  basic  statistics  for  the  remOverArrRecRequestsRatio 
(removed  over  arrived  reconnaissance  requests  ratio).  With  the  realistic  values  the 
authors  used,  the  mean  of  remOverArrRecRequestsRatio  is  0.41  with  a  standard 
deviation  of  0.08.  Distribution  of  remOverArrRecRequestsRatio  is  unimodal  and 
symmetric.  Maximum  remOverArrRecRequestsRatio  is  0.65,  which  is  big. 


re  mo  ved  Ove  r  Ar  r  RecReque  sts  Ratio 


□□ . 

1 _ 1 

n-TTrdT 

TTitf^ 

0.2  0.3  OA 

\  0.5  0.6 

Quantiles 


100.0% 

maximum 

0.65579 

99.5% 

0.61735 

97.5% 

0.58427 

90.0% 

0.52365 

75.0% 

quartile 

0.46945 

50.0% 

median 

0.41615 

25.0% 

quartile 

0.36255 

10.0% 

0.31239 

2.5% 

0.24981 

0.5% 

0.18659 

0.0% 

minimum 

0.12827 

Moments 

0.4161831 
0.0828509 
0.0011717 
0.4184801 
0.413886 
5000 


Mean 
Std  Dev 
Std  Err  Mean 
Upper  95%  Mean 
Lower  95%  Mean 
N 


Figure  96.  Distribution  of  remOverArrRecRequestsRatio 


b.  Regression  Model  Built  by  Using  Only  Main  Effects 


After  using  a  stepwise  regression  analysis  with  only  the  main  effects  of 


twenty-one  input  factors,  a  regression  model  for  the  remOverArrRecRequestsRatio  was 
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built.  Figure  97  shows  the  aetual  by  predicted  plot  of  this  model.  The  p- value  of  this 
model  is  less  than  0.0001,  which  means  the  model  is  statistically  significant.  The 
value  of  0.80  indicates  that  80%  of  the  variability  in  the  response  variable  is  explained  by 
the  model. 


Figure  97.  Actual  by  Predicted  of  Model  for  remOverArrRecRequestsRatio 


Figure  98  shows  the  Summary  of  Fit  (SoF)  and  Analysis  of  Variance 
(AoV)  tables  for  the  regression  model  built  for  remOverArrRecRequestsRatio. 


Summary  of  Fit 

RSquare  0.799341 

RSquare  Adj  0.798818 

Root  Mean  Square  Error  0.037161 
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Prob  >  F 
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0.0000* 

Figure  98.  SoF  and  AoV  Tables  of  Model  for  remOverArrRecRequestsRatio 


Figure  99  shows  thirteen  statistically  significant  terms,  sorted  by  their 
importance  on  response.  Mode  time  of  pods  repair,  mode  number  of  reconnaissance 
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requests,  probability  of  aireraft  intereeption,  probability  of  both  type  1  and  2  pods 
defects,  probability  of  bad  weather  condition,  mode  time  of  aircraft  repair  and  half 
distance  of  time  for  pods  repair  have  a  positive  effect  on  remOverArrRecRequestsRatio. 


On  the  other  hand,  the  number  of  type  2  pods,  number  of  type  1  pods, 
probability  of  vertical  angle  type,  half  distance  aircraft  repair  time  and  probability  of  data 
link  defect  have  a  negative  effect  on  remOverArrRecRequestsRatio.  The  half  distance 
aircraft  repair  time  and  probability  of  data  link  defects  negative  effects  seem  counter¬ 
intuitive,  although  these  have  much  smaller  impacts  than  the  other  terms  with  negative 
effects. 
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0.009013 

-2.948e-5 

4.507e-6 

4.8893G-6 

8.348G-7 

2.0848e-5 

4.506e-6 

-0.037634 

0.009013 
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Figure  99.  Sorted  Parameter  Estimates  of  Model  for  remOverArrRecRequestsRatio 
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Figure  100.  Residual  by  Predicted  Plot  of  Model  for  remOverArrRecRequestsRatio 
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F igure  101.  Residuals  of  Model  for  remOverArrReeRequestsRatio 

As  seen  from  Figure  100,  the  vertical  spread  of  the  residuals  is  almost  the 
same.  A  model  that  includes  some  interactions  or  quadratic  effects  may  provide  a  better 
fit.  As  seen  from  Figure  101,  the  distribution  of  residuals  is  bimodal  but  not  symmetric. 

c.  Regression  Model  Built  by  Using  Main  Effects,  Interactions  and 
Quadratic  Effects 

After  using  a  stepwise  regression  analysis  with  main  effects,  interactions 
and  quadratic  effects  of  twenty-one  input  factors,  a  regression  model  for  the 
remOverArrRecRquestsRatio  was  built.  Figure  102  shows  the  actual  by  predicted  plot  of 
this  model.  The  p-value  of  this  model  is  less  than  0.0001,  which  means  the  model  is 
statistically  significant.  The  R^  value  of  0.90  indicates  that  90%  of  the  variability  in  the 
response  variable  is  explained  by  the  model. 
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Actual  by  Predicted  Plot 


Predicted  P<.0001  RSq=0.90 
RMSE=0.0262 


Figure  102.  Actual  by  Predicted  Plot  of  Model  for  remOverArrRecRequestsRatio 


Figure  103  shows  Summary  of  Fit  (SoF)  and  Analysis  of  Variance  (AoV) 
tables  of  regression  model  built  for  remOverArrRecRequestsRatio. 

Summary  of  Fit 

RSquare  0.900394 

RSquareAdj  0.899953 

Root  Mean  Square  Error  0.026206 

Mean  of  Response  0.416183 

Observations  (or  Sum  Wgts)  5000 

Analysis  of  Variance 


Sum  of 


Source 

DF 

Squares 

Mean  Square 

F  Ratio 

Model 

22 

30.896580 

1.40439 

2044.988 

Error 

4977 

3.417941 

0.00069 

Prob  >  F 

C.  Total 

4999 

34.314521 

0.0000* 

Figure  103.  SoF  and  AoV  Tables  of  Model  for  remOverArrRecRequestsRatio 


Figure  104  shows  twenty-two  statistically  significant  terms,  sorted  by  their 
importance  on  response.  Figure  105  shows  interactions  between  the  significant  terms. 
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Sorted  Parameter  Estimates 


Term 

Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

mode  Pod  s  Re  pTi  m  e 

5.5515e-5 

6.191e-7 

89.68 

0.0000* 

modeNumReq 

0.0157178 

0.000182 

86.60 

1 

0.0000* 

numT2Pods 

-0.025662 

0.000308 

-83.38 

1 

0.0000* 

tlPodsDefProb 

0.9863369 

0.014166 

69.63 

\ 

0.0000* 

aCInterProb 

0.9538715 

0.014634 

65.18 

1 

0.0000* 

(aClnterProb-0.075)*(aClnterProb-0.075) 

-33.28651 

0.657934 

-50.59 

1 

0.0000* 

badWeatConProb 

0.302247 

0.006564 

46.05 

1 

0.0000* 

verAngTProb 

-0.271771 

0.006514 

-41.72 

1 

0.0000* 

t2PodsDefProb 

0.5375632 

0.013069 

41.13 

1 

<.0001* 

numTlPods 

-0.009163 

0.000245 

-37.33 

L 

<.0001* 

(modeNumReq- 12. 89)*(aClnterProb-0.075) 

-0.160204 

0.006861 

-23.35 

L 

<.0001* 

(numTlPods-8)*(aClnterProb-0.075) 

0.2173503 

0.009353 

23.24 

<.0001* 

(numT2Pods-4.5)*(aClnterProb-0.075) 

0.2195525 

0.011729 

18.72 

<.0001* 

(numT2Pods-4.5)*(verAngTProb-0.5) 

0.0840791 

0.005278 

15.93 

<.0001* 

(numTlPods-8)*(verAngTProb-0.5) 

-0.065634 

0.004209 

-15.59 

<.0001* 

(modeNumReq- 12. 89)*(modeNumReq-12. 89) 

-0.001528 

0.000111 

-13.76 

<.0001* 

(tlPodsDefProb-0.1)*(aClnterProb-0.075) 

6.9444003 

0.53184 

13.06 

<.0001* 

(modePodsRepTime-2520)*(aClnterProb-0.075) 

0.0002805 

2.354e-5 

11.91 

<.0001* 

(modePodsRepTime-2520)*(t2PodsDefProb-0.1) 

0.0002095 

1.983e-5 

10.57 

<.0001* 

(numTlPods-8)*(t2PodsDefProb-0.1) 

-0.082234 

0.008418 

-9.77 

<.0001* 

(modeNumReq-12.89)*(modePodsRepTime-2520) 

2.9428e-6 

3.184e-7 

9.24 

<.0001* 

upRepReqProb 

0.0161007 

0.006573 

2.45 

0.0143* 

Figure  1 04.  Sorted  Parameter  Estimates  of  Model  for  remOverArrRecRequestsRatio 
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Figure  105.  Interaction  Profiler  of  Model  for  remOverArrRecRequestsRatio 
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The  first  remarkable  interaetion  is  the  one  between  the  number  of  type  1 
pods  and  vertical  angle  image  type  probability.  For  the  six  Type  1  pods  (type  1  pods 
takes  vertical  imagery  only),  an  increase  in  the  vertical  angle  image  type  probability 
decreases  the  remOverArrRecRequestsRatio  a  little.  However,  for  ten  Type  1  pods,  an 
increase  in  the  vertical  angle  image  type  probability  decreases  the 
remOverArrRecRequestsRatio  a  lot.  We  can  conclude  that  we  need  more  than  six  Type  1 
pods  to  increase  its  negative  effect  on  remOverArrRecRequestsRatio. 

Another  interesting  interaction  is  the  one  between  the  number  of  type  1 
pods  and  aircraft  interception  probability.  For  the  0.12  interception  probability,  an 
increase  in  the  number  of  type  1  pods  does  not  change  the  remOverArrRecRequestsRatio. 
However,  for  the  0.03  interception  probability,  an  increase  in  the  number  of  type  1  pods 
decreases  the  remOverArrRecRequestsRatio  a  lot.  We  can  conclude  that  we  need  more 
aircraft  to  satisfy  reconnaissance  requests  when  the  aircraft  interception  probability  is 
high.  For  a  low  aircraft  interception  probability,  we  can  see  the  negative  effect  of  the 
number  of  type  1  pods. 
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Figure  106.  Residual  by  Predicted  Plot  of  Model  for  remOverArrRecRequestsRatio 


As  seen  from  Figure  106,  the  vertical  spread  of  the  residuals  is  the  same, 
which  is  good.  As  seen  from  Figure  107,  the  distribution  of  residuals  is  symmetric  and 
unimodal,  which  is  also  good. 
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Figure  107.  Residual  of  Model  for  remOverArrRecRequestsRatio 
d.  Partition  Tree  Model 

Figure  108  shows  a  partition  tree  example  with  only  six  splits.  The  R 
value  of  0.542  indicates  that  54%  of  the  variability  in  the  response  variable  is  explained 
by  the  model.  More  splits  could  be  added  to  this  tree  if  future  analysts  wish  to  examine 
smaller  subsets  of  the  data  in  more  detail  and  to  obtain  more  explanatory  power  from 
model.  The  particular  areas  of  interest  in  the  tree  are  the  leftmost  and  rightmost  areas.  In 
the  leftmost  area  we  see  that  when  mode  time  for  pods  repair  is  less  than  3,250,  the 
number  of  type  2  pods  is  greater  than  or  equal  to  five,  and  aircraft  interception  probability 
is  greater  than  or  equal  to  0.05,  and  mode  number  of  requests  is  less  than  fourteen, 
remOverArrRecRequestsRatio  is  at  minimum  (Mean  0.36  with  a  SD  0.05).  In  the 
rightmost  area  we  see  that  when  the  mode  time  for  pods  repair  is  greater  than  or  equal  to 
3,250,  and  type  1  pod  defect  probability  is  greater  than  or  equal  to  0.09, 
remOverArrRecRequestsRatio  is  at  maximum  (Mean  0.54  with  a  SD  0.04). 
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Figure  108.  Partition  Tree  Model  for  remOverArrRecRequestsRatio 


2.  Robust  Configuration  for  Reconnaissance  Squadron 

After  using  a  stepwise  regression  analysis  with  main  effects,  interactions  and 
quadratic  effects  of  five  decision  factors,  regression  models  for  mean  and  standard 
deviations  of  remOverArrRecRequestsRatio  and  avgTimeAtWFForResRecRequests  were 
built.  The  remOverArrRecRequestsRatio  MoE  is  more  important  than 
avgTimeAtWFForResRecRequests  MoE  for  the  reconnaissance  squadron  commander. 
Therefore  we  will  deal  with  models  for  remOverArrRecRequestsRatio  first.  If  there  will 
be  any  conflicting  decision  factor  setting(s),  the  one(s)  used  for 
remOverArrRecRequestsRatio  will  be  recommended.  Figure  109  shows  the  prediction 
profiler  for  the  model  of  remOverArrRecRequestsRatio ’s  standard  deviation.  Relevant 
decision  factors  were  set  to  minimize  the  standard  deviation  of 
remOverArrRecRequestsRatio,  and  these  same  values  were  used  to  minimize  mean  of 
remOverArrRecRequestsRatio,  as  seen  in  Figure  110. 
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Figure  109.  Prediction  Profiler  for  Model  of  remOverArrRecRequestsRatio’s  Standard 

Deviation 
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Figure  110.  Prediction  Profiler  for  Model  of  remOverArrRecRequestsRatio’s  Mean 

Figure  111  shows  the  prediction  profiler  for  the  model  of 
avgXimeAtWFForResRecRequests’s  standard  deviation.  Relevant  decision  factors  were 
set  to  minimize  standard  deviation  of  avgTimeAtWFForResRecRequests,  and  these  same 
values  were  used  to  minimize  mean  of  avgTimeAtWFForResRecRequests,  as  seen  in 
Figure  112. 
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Figure  111.  Prediction  Profiler  for  Model  of  avgTimeAtWFForResRecRequests’s 

Standard  Deviation 
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Figure  1 12.  Prediction  Profiler  for  Model  of  avgTimeAtWFForResRecRequests’s  Mean 


After  comparing  decision  factors’  settings  for  remOverArrRecRequestsRatio  and 
avgTimeAtWFForResRecRequests  MoEs,  we  see  that  there  is  only  one  conflicting 
decision  factor  setting,  which  is  the  numType2Pods.  It  was  set  to  three  to  minimize  the 
standard  deviation  of  remOverArrRecRequestsRatio,  and  this  value  is  recommended.  The 
decision  factors  not  seen  in  prediction  profilers  for  either  remOverArrRecRequestsRatio 
or  avgTimeAtWFForResRecRequests  are  set  to  their  minimum  levels  used  in  experiment. 
Table  7  shows  the  robust  configuration  for  the  reconnaissance  squadron  in  configuration: 
F-16  /  situation:  war  scenario. 
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Table  7. 


Number 

Decision  Factors 

Value 

1 

numTlPods 

10 

2 

numT2Pods 

3 

3 

nuniACs 

20 

4 

numPilots 

35 

5 

numAnalysts 

5 

Robust  Configuration  for  Reconnaissance  Squadron  in  Configuration:  F- 
16  /  Situation:  War  Scenario 
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VI.  RECONNAISSANCE  SQUADRON  WORKFLOW 
SIMULATION  GUI  FOR  QUICK  ANALYSIS 


There  is  one  Graphieal  User  Interfaee  (GUI)  for  eaeh  seenario.  In  this  thesis  there 
were  in  total  four  scenarios,  thus  four  GUIs.  The  GUIs  are  for  facilitating  quick  analysis 
by  the  user.  The  user  can  enter  the  most  updated  information  for  input  factors  of  each 
scenario.  Figure  113  shows  Peace  Situation  Configuration  RF-4  GUI,  which  consists  of 
Inputs  and  Statistics  panes.  The  input  pane  contains  four  tabs:  Sim  Params,  Params(l), 
Params(2),  and  Probabilities.  The  Sim  Params  tab  includes  replication  and  duration  input 
parameters,  while  the  Params(l),  Params(2)  and  Probabilities  tabs  includes  model- 
specific  input  parameters,  such  as  the  number  of  aircraft,  number  of  pilots,  probability  of 
aircraft  defect,  etc.  Figure  1 14  shows  War  Situation  Configuration  RF-4  GUI.  In  contrast 
to  the  Peace  Situation  Configuration  RF-4  GUI,  it  includes  the  probability  of  aircraft 
interception  input  and  excludes  input  parameters  for  priority  due  dates. 


Figure  113.  Conf :  RF-4  /  Sit.:  Peace  Quick  Analysis  GUI 
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The  statistics  pane  is  used  for  simulation  results.  After  the  simulation  run,  the 
number  of  reconnaissance  requests,  number  of  responded  requests,  number  of  removed 
requests,  and  average  time  at  workflow  for  both  day  and  night  requests,  etc.,  are 
presented  with  a  95%  confidence  interval  (Cl).  For  war  situations,  the  statistics  pane  also 
gives  the  average  time  that  the  RF-4  reconnaissance  squadron  was  operational. 


Figure  1 14.  Conf :  RF-4  /  Sit.:  War  Quick  Analysis  GUI 

Figure  115  shows  the  Peace  Situation  Configuration  F-16  GUI,  with  model 
specific  input  parameters  such  as  data  link  defect  probability,  night  mission  probability, 
requiring  update  reconnaissance  mission  probability,  etc. 
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Figure  115.  Conf.:  F-16  /  Sit.:  Peace  Quick  Analysis  GUI 

Figure  1 16  depicts  the  War  Situation  Configuration  F-16  GUI  for  quick  analysis. 
This  GUI  is  different  than  the  Peace  Situation  Configuration  F-16  GUI  depicted  in  Figure 
115  because  it  includes  the  probability  of  aircraft  interception  input,  and  excludes  priority 
due  dates  input  parameters  specific  to  the  peace  scenario  (recall  that  all  wartime  requests 
are  due  within  one  day). 
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Figure  116.  Conf.:  F-16  /  Sit.:  War  Quick  Analysis  GUI 

There  are  four  buttons  at  the  bottom  of  the  GUI.  The  default  button  sets  default 
values  defined  in  the  simulation  for  input  parameters;  this  way,  users  are  not  required  to 
change  every  input  parameter.  The  run  button  starts  the  simulation.  It  also  checks  the 
parameters  for  correct  input  types.  Moreover,  when  run  is  clicked,  a  message  box  appears 
which  says  the  simulation  is  running.  The  message  pane  closes  automatically  once  the 
results  of  the  simulation  are  written  in  the  statistics  pane.  The  cancel  button  closes  the 
GUI,  while  the  save  button  saves  the  results  of  the  simulation  in  a  user-defined  name  and 
location. 
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VII.  CONCLUSIONS  AND  FUTURE  WORK 


A.  CONCLUSIONS 

A  simplified  reconnaissance  cycle  includes  the  arrival  of  reconnaissance  requests, 
planning  of  reconnaissance  flights,  flying  the  mission  and  exploitation  of  the  films  or 
images,  and  then  dissemination  of  the  intelligence  reports.  The  reconnaissance  cycle  was 
modeled  for  four  different  scenarios  (peace  and  war  as  situations,  and  RF-4  and  F-16  as 
configurations).  There  are  two  points  of  view  about  reconnaissance  squadron  workflow. 
The  first  one  is  the  reconnaissance  requesters’  view.  They  want  to  know  the  estimated 
time  it  would  take  for  a  request  to  be  answered  based  on  the  resources  and  other  factors 
before  the  actual  request  was  made.  The  second  one  is  the  reconnaissance  squadron 
commanders’  perspective.  They  want  to  respond  to  as  many  reconnaissance  requests  as 
possible.  For  that  reason,  they  want  to  know  and  revise  the  ideal  numbers  of  personnel 
and  equipment.  For  the  purpose  of  answering  these  questions,  satisfying  these  requests, 
and  having  a  better  understanding  about  reconnaissance  cycle,  we  modeled  and  analyzed 
Reconnaissance  Squadron  Workflow  for  four  different  scenarios. 

For  the  development  of  models,  we  used  a  basic  modeling  process  and  discrete 
event  simulation  techniques.  Simkit  was  used  as  the  simulation  tool.  Input  and 
distributional  parameters  are  based  on  realistic  notional  data  (because  of  their  classified 
nature)  that  are  used  to  generate  the  levels  of  each  input  factor.  Two  designs  (one  for 
decision  and  one  for  noise  factors)  were  generated  for  each  model  by  using  the 
“Generating  and  Improving  Orthogonal  Designs  by  Using  Mixed  Integer  Programming 
tool.’’  (see  also  Vieira  et  al.,  2010)  These  two  designs  were  crossed  to  generate  designs 
for  each  scenario.  We  experimented  all  of  the  four  models  by  using  these  designs. 
Generated  output  data  was  imported  to  the  statistical  analysis  software,  IMP,  for  analysis. 
Effective  regression  models  were  generated  to  estimate  and  analyze  each  MoE.  The 
factors  that  have  the  most  significant  effects  on  the  MoEs  were  identified  in  relevant 
sections.  We  found  a  robust  configuration  (set  of  decision  factor  settings)  for  each 
scenario,  in  order  to  identify  ideal  numbers  of  personnel  and  equipment. 
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In  Chapter  V,  we  provided  detailed  explanations  of  how  the  factors  affect  the 
MoEs.  When  all  regression  models  are  considered,  we  see  that  there  is  no  common  rule 
to  determine  which  factors  (either  decision  or  noise)  are  the  key  determinants  of  each 
MoE.  But  we  noticed  that  the  ratio  of  the  total  number  of  noise  factors  to  the  total  number 
of  decision  factors  in  each  model  is  high.  Some  of  these  noise  factors  could  be 
controllable,  including  aircraft,  camera  and  pod  defect  probabilities  and  their  repair  times. 
Some  precautionary  measures  should  be  taken  to  reduce  these  defect  probabilities  and 
repair  times. 

Specifically,  in  the  RF-4  configuration  models,  pilot  filming  error  is  a  significant 
factor,  which  shows  that  training  of  the  pilots  is  also  important  and  cannot  be  ignored. 

When  the  F- 16  models  are  considered,  we  see  that  data  link  defect  probability  is  a 
significant  factor.  This  suggests  that  data  link  capability  is  an  important  factor  in 
providing  reconnaissance  requests  quickly,  and  special  precautions  should  be  taken  to 
keep  this  capability  working. 

We  also  developed  four  GUIs — one  for  each  model  (scenario) — to  facilitate  quick 
analysis.  Future  users  can  run  each  simulation  extremely  fast  with  correct  and  valid 
inputs.  When  the  simulation  ends,  some  of  the  average  values  for  state  variables  such  as 
number  of  arrived  requests,  responded  requests,  un-responded  requests,  average  time  at 
workflow  for  responded  requests,  etc.,  are  presented  in  95%  Cl  intervals;  these  can  be 
used  for  defense  resource  planning,  resource  management,  and  decision  making. 

There  is  optimization  involved  in  flight  planning  for  reconnaissance  missions. 
When  a  flight  needs  to  be  planned,  simulation  assigns  the  resources  that  will  be  available 
at  the  earliest  time. 

B.  FUTURE  WORK 

In  the  developed  models,  reconnaissance  requests  have  fixed  target  assignments 
and  are  assumed  to  arrive  to  the  squadron  in  bulk.  Our  model  assumes  that 
reconnaissance  requests  arrive  to  the  squadron  the  day  before  the  actual  mission  takes 
place.  However,  in  real  world  situations,  there  are  also  emerging  targets  to  be  filmed, 
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such  as  mobile  units.  So,  in  addition  to  the  bulk  arriving  reconnaissance  requests, 
dynamic  arrival  of  reconnaissance  requests  can  be  implemented.  Also,  flight  leg  planning 
integrated  with  air  refueling  can  be  implemented  to  represent  flight  legs  more 
realistically. 

2D  or  3D  visualization  can  be  added  to  overall  simulation  in  compliance  with 
DBS.  Real  time  statistics  can  be  generated  while  the  simulation  is  running.  With  these 
additions,  future  users  can  get  a  better  understanding  about  the  workload  and  workflow  of 
the  reconnaissance  squadron. 

Pilots  and  image  analysts’  sick  days  or  daily  leaves  can  be  implemented  in  each 
model.  These  factors  might  change  the  regression  models  and/or  increase  the  number  of 
pilots  and  image  analysts  in  the  recommended  robust  configurations. 
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APPENDIX.  INPUT  FACTOR  PARAMETERS  FOR  DIFFERENT 

SCENARIOS 


A.  RF-4  CONFIGURATION  /  PEACE  SITUATION  SCENARIO 


nu  mil  Cams 

numT2Cam5 

numT3Cam5 

numT4Cam5 

10 

5 

4 

3 

15 

S 

10 

S 

B  Decision  Factors 

nunnT5Canns 

numACs 

numPitots 

numAnalysts 

2 

20 

35 

5 

6 

25 

45 

10 

rmodeNuimReq 

halfQistNumReq 

modeHighPriDueDays 

h  a  If  □  istH  igh  P  r  i  □  ue  □  ays 

10 

2 

4 

1 

16 

6 

6 

3 

pri  Due  Days 
LowOver  High  Ratio 

□plmgTProb 

tlCamsDefProb 

t2CamsDefProb 

1.2 

0.75 

0.15 

0.15 

1.6 

0.9 

0.55 

0.55 

19  Noise  Factors 

tBCannsDefProb 

t4CamsDefProb 

t5CamsDefProb 

aCDefProb 

0.15 

0.2 

0.15 

0.1 

-1 

1 

0.55 

0.5 

0.55 

0.3 

pilotFIlErProb 

badWeatConProb 

modeACRepTime 

halfDistACRepTime 

0.1 

0.1 

14401 

500 

0.3 

0.3 

3600 

900 

rmodeTacCams 

Re  pit  me 

halfDistTacCams 

RepTime 

camsRepTime 

StraOverTacRatio 

90 

45 

1.4 

120 

60 

l.S 
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numlCanns  nunnT2Cann5  numTBCanns  numT^Canns 

1 

1 

10 

5 

4 

3 

15 

8 

10 

8 

1 

[ 

1 

8  Dec  151  on  Factors 

nuniTBCams  numACs  numPilots  numAnaly^ts 

1 

1 

2 

20 

35 

5 

6 

25 

45 

10 

modeNumReq 

halfDistNumReq 

aCIlnterceptProb 

optlrnglProb 

n 

10 

2 

0.05 

0.75 

16 

6 

0.2 

0.9 

tlCamsQefProb 

t2CannsDefProb 

t3CamsDefProb 

t4CamsDefProb 

0.15 

0.15 

0.15 

02 

0.55 

0.55 

0.55 

0.6 

17  Noise  Factors 

t5Canfi5DefProb 

aCDefProb 

pilotFilErProb 

badWeatConProb 

0.15 

0.1 

0.1 

01 

0.55 

0.3 

0.3 

0.3 

modeACRepTinne 

h  a  If  D  istACRepTi  me 

m  odela  c  Ca  a  c  Ca 

RepTime  RepTime 

1440 

500 

90  45 

3600 

900 

120 

60 

camsRepTinne 
StraOverTac  Ratio  H 

1.4 

l.S 
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F-16  CONFIGURATION  /  PEACE  SITUATION  SCENARIO 


nurmTlPods  nunnT2Pod5  numACs  nurmPilots 

1 

1 

6 

3 

20 

35 

10 

6 

25 

45 

1 

1 

5  Decision  Factors 

nurmAnalysts 

1 

1 

5 

10 

rmodeNunnReq  halfDistNumPeq  imodeHighPri  Due  Days  halfDistHighPriDueDays 

10 

2 

4 

1 

16 

6 

6 

3 

pri  Due  D  ays 

LowO  ver  HlBh  Ratio 

nnodeACRepTime 

halfDistACRepTinne 

nnodePod5 

RepTirre 

1.2 

1440 

500 

1440 

1.6 

3600 

900 

3600 

halfDiStPodS  ,  -.-rv  U  -  -.-rv  U  -  U 

□plrnglProb  verAngTPrQb  nightFhghtProb 

RepTime  B 

IS  Noise  Factors 

500 

0.4 

0.4 

O.S 

900 

0.6 

0.6 

0.9 

badWeatConProb  dataLinkDefProb  tlPodsDetProb 

t2  Pods  DetP  rob  ■ 

0.1 

0.1 

0.05 

0.05 

0.3 

0.3 

0.15 

0.15 

aCDefProb 

u  p  Re  p  Re  q  P  r  □  b*^B 

0.05 

0.1 

0.15 

0.3 
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numTlPods  nunnT2Pods  numACs  numPilots 

6 

3 

20 

35 

10 

6 

25 

45 

5  Decision  Factors 

numAnalysts 

5 

10 

inodGlMunnRGq  halfDistNuinRGq  modGACRGpTirriG  halfDistACRGpTiiriG^^* 

10 

2 

1440 

500 

16 

6 

3600 

900 

modGPods  halfDistPods 

RGpTinnG  RGpTinnG 

opImgTProb 

VGrAngTProb 

1440 

500 

0.4 

0.4 

3600 

SOO 

0.6 

0.6 

■— 

16  Noise  Factors 

nightFlightProb  badWGatConProb  dataLinkDGfProb^^K 

tip  0  ds  D  Gf  P  ro 

0.8 

0.1 

0.1 

0.05 

O.S 

0.3 

0.3 

0.15 

tZPodsDGfProb  aCOGfProb  aCIntGrProb  upRGpRGqProb 

0.05 

0.05 

0.03 

0.1 

0.15 

0.15 

0.12 

0.3 
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